Can negotiated MTU w/ MCUMgr params work for serial (uart/usb/shell transport) #73

Activity

1. JPHutchins

   added

   bugSomething isn't workingSomething isn't working

   documentationImprovements or additions to documentationImprovements or additions to documentation

   on Aug 9, 2025
2. JPHutchins

   added the

   Task

   issue type on Aug 9, 2025
3. JPHutchins

   added

   enhancementNew feature or requestNew feature or request

   help wantedExtra attention is neededExtra attention is needed

   on Aug 9, 2025
4. JPHutchins commented on Aug 9, 2025

   

   JPHutchins

   on Aug 9, 2025

   CollaboratorAuthor

   More actions

   Sorta related, since this work will enable users of smpclient (e.g. smpmgr) to provide easy access to transport-specific configuration: #58
5. joerchan commented on Sep 12, 2025

   

   joerchan

   on Sep 12, 2025

   Last edited by joerchan

   More actions

   I've done some investigation into this issue and this is what I have concluded.

   MTU for uart/usb/shell should be assumed to be 127 as this is a limit that is not always imposed
   by the implementation, but is there in the specification.
   In zephyr this is only enforced by the SHELL transport.
   See this PR for failed attempt to remove this limitation:
   zephyrproject-rtos/zephyr#95891.

   That means that the following configuration is invalid: dutfirmware/usb_smp_dut_1024_1_1024.conf:

   # ~31.61 KB/s
   CONFIG_UART_MCUMGR_RX_BUF_SIZE=1024
   CONFIG_UART_MCUMGR_RX_BUF_COUNT=1
   CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE=1024.
   

   It is however possible to get the same throughput with a different configuration.

   Serial transport initialized with:

   max_smp_encoded_frame_size = 2048
   line_length = 128
   line_buffers = 16
   

   config for SHELL transport

   # ~31.3 KB/s
   CONFIG_MCUMGR_TRANSPORT_NETBUF_COUNT=2
   CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE=2048
   # Non-functional config, included for demonstration, MTU is always 127
   CONFIG_MCUMGR_TRANSPORT_SHELL_MTU=127
   CONFIG_MCUMGR_TRANSPORT_SHELL_RX_BUF_COUNT=16
   

   config for UART transport

   # Not tested, but assume speed would be the same, as it is sent the same from smpmgr
   # The actual MTU of the UART transport
   CONFIG_UART_MCUMGR_RX_BUF_SIZE=127
   CONFIG_UART_MCUMGR_RX_BUF_COUNT=16
   CONFIG_MCUMGR_TRANSPORT_NETBUF_COUNT=2
   CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE=2048
   # Unused config, but should match the MTU.
   CONFIG_MCUMGR_TRANSPORT_UART_MTU=127
   

   I think the best thing for smpmgr would be to:

   1. Assume an MTU of 127 by default.
   2. Read the MCUMGR_PARAM page for line_length.
   3. Calculate line_buffers as line_length / MTU.
   4. Allow giving line_buffers / line_length to override the default behavior, together with MTU.
      That way all assumptions can be overridden.

   Without changing the line_buffers setting, and using the enforced shell MTU of 127 I can only
   get around 7.2 KB/s using the default smpclient, and only 2.3 UB/s when setting --mtu 127.

   The reason for line_buffers being so important for serial / UART is that smpmgr sends all
   frames back to back, while there is an around 15 ms delay between each SMP-packet. This delay
   I don't know where comes from but seems fixed, and is not affected by baudrate.
   That is why both 1M and 230.4K will have the same throughput.

   Happy to contribute, or aid in testing.
   Let me know if I should provide any more information.

   👍React with 👍1Reacted by JP Hutchins
6. JPHutchins commented on Sep 16, 2025

   

   JPHutchins

   on Sep 16, 2025

   Last edited by JPHutchins

   CollaboratorAuthor

   More actions

   This is great! Discovering that small line length can still allow for fast transfers could resolve the issues with mcumgr param negotiation for the serial transports.

   I think the best thing for smpmgr would be to:

   1. Assume an MTU of 127 by default.
   2. Read the MCUMGR_PARAM page for line_length.
   3. Calculate line_buffers as line_length / MTU.
   4. Allow giving line_buffers / line_length to override the default behavior, together with MTU.
      That way all assumptions can be overridden.

   1. For serial, the SMP "MTU" is called "line length". This is the maximum transmission unit of encoded smp frames. It's important to agree on the terminology since much of the confusion stems from the vocabulary and dead or poorly named configs in Zephyr.
   2. The MCUMGR_PARAM does not give line length. It gives the netbuf size and netbuf count, where netbuf size can be "max unencoded smp frame size". It is never necessary or useful to have more than one netbuf. Zephyr is not making compile time guarantees that the netbuf size matches the line length. I will need to look at the Zephyr impl, but I'm assuming that it needs

   • 127 byte buffer to hold the encoded packet - a "line" - I think it's called this because the \n is used as delimiter 🙄
   • netbuf size buffer to hold the incrementally decoded smp packet
     I do agree with the approach: line_buffers = netbuf_size // 127

   3. I like where this is going
   4. Agreed!

   My only concern is that it seems that the 16 buffer config uses 2x the RAM allocation for the same speed. I may be wrong, since the 1024 config might allocate 2048 as well - I need to investigate Zephyr impl to understand what it's doing with these.

   The primary fix, for a default of 127, would be upstream: https://github.com/JPHutchins/smp/blob/edb8a012d8e9fa5b85e524296b73f8e44c0ecdae/smp/packet.py#L30-L31

   Finally, this also needs to be tested with MCUBoot's SMP server implementation: https://github.com/mcu-tools/mcuboot/blob/main/boot/boot_serial/src/boot_serial.c
7. joerchan commented on Sep 18, 2025

   

   joerchan

   on Sep 18, 2025

   Last edited by joerchan

   More actions

   It has been quite confusing to understand and to figure everything out. The documentation was not very helpful and i basically had to dive into the implementation to understand the full picture.
   I did actually mean the MCUMGR_PARAM buf_size and buf_count.

   So to rewrite my suggestion with correct terminology:

   1. Assume an line_length of 127 by default.
   2. Read the MCUMGR_PARAM page for buf_size.
   3. Calculate line_buffers as buf_size / line_length.
   4. Allow giving line_buffers / line_length to override the default behavior, along with buf_size (max_unencoded_packet_size).
      That way all assumptions can be overridden.

   My only concern is that it seems that the 16 buffer config uses 2x the RAM allocation for the same speed. I may be wrong, since the 1024 config might allocate 2048 as well - I need to investigate Zephyr impl to understand what it's doing with these.

   There is still the possebility to do the other config that is the fastest for UART, for the shell however there is no choice unless you patch zephyr to remove this limit yourself.
   Unfortunately it seems that you would need the 2048 config, as using 1024 only gives you around 20 KB/s.

   You actually don't need the full 16 RX buffers in the MCU though, as you can just have 4 and then rely on the buffers being processed fast enough to always have available buffers, but if you run out the transfer could fail. So 16 is needed to have guaranteed numbers of buffers available.

   🚀React with 🚀1Reacted by JP Hutchins
8. JPHutchins

   added a commit that references this issue on Oct 12, 2025

   fix: default line length 8192 -> 127

   ...

   985e66f
9. JPHutchins commented on Oct 12, 2025

   

   JPHutchins

   on Oct 12, 2025

   CollaboratorAuthor

   More actions

   So to rewrite my suggestion with correct terminology:

   1. Assume an line_length of 127 by default.
   2. Read the MCUMGR_PARAM page for buf_size.
   3. Calculate line_buffers as buf_size / line_length.
   4. Allow giving line_buffers / line_length to override the default behavior, along with buf_size (max_unencoded_packet_size).
      That way all assumptions can be overridden.

   I will implement in this manner, thank you!
10. JPHutchins

    self-assigned this

    on Oct 12, 2025
11. JPHutchins

    added a commit that references this issue on Jun 12, 2026

    Merge pull request #81 from intercreate/feature/#73/serial-fragmentat…

    ...

    f583933
12. JPHutchins

    closed this as completedon Jul 14, 2026