Are intel hex files converted into a binary format before they are written to Atmega's flash?

Question

I have been going through some resources on bootloaders, programmers, etc (I'm very new to this specific subject) and how they work with the Atmega chips and Arduino board programming. My research may have been shallow but none of the online articles/blogs/forum posts that I have been through mentioned that the .hex file (in intel hex ASCII format) needs to be converted to any other kind of binary format. It did seem odd to me though that one would directly write ASCII plain text into an MCU's flash. Also, when using AVR Burn-o-Mat (a GUI front end for avrdude) I did notice a couple of times that the number of bytes being flashed was significantly less than the actual size of the hex file. Most recently I went through this library on github which converts intel hex into a binary format. So am I right in saying that the hex file needs to be converted to binary? So for instance, does avrdude basically do this before sending the binary stream to the bootloader (when uploading a sketch via the IDE)?

Nick Gammon · Answer 1 · 2023-06-21T22:33:50.473

Yes, the hex codes in the .hex file are read in by avrdude (or other programs) and written to the MCU in binary. A small microprocessor would not be able to read ASCII format commands.

See my answer about What happens when code is uploaded using the bootloader?

(Following on from some comments)

I did notice a couple of times that the number of bytes being flashed was significantly less than the actual size of the hex file

I think there is some confusion here, because of the different way numbers can be represented, or stored. Just for example, the number which we know as "twenty" could be written (in a file) as:

20 (decimal)
00020 (decimal with leading zeroes)
0014 (hex)
0024 (octal)
10100 (binary digits)

Now a program like Avrdude could read numbers in any of those formats, internally convert them to "binary" and send a single byte, being that binary number, to the MCU for placing into a program memory byte.

The .hex format, which the Arduino system produces as part of the compilation process, uses lines in this format:

Line format:
  :nnaaaatt(data)ss
  Where:
  :      = a colon
  (All of below in hex format)
  nn     = length of data part
  aaaa   = address (eg. where to write data)
  tt     = transaction type
           00 = data
           01 = end of file
           02 = extended segment address (changes high-order byte of the address)
           03 = start segment address *
           04 = linear address *
           05 = start linear address *
  (data) = variable length data
  ss     = sumcheck
            * We don't use these

The above from comments in my file https://github.com/nickgammon/arduino_sketches/blob/master/Atmega_Hex_Uploader/File_Utils.ino

Example: Optiboot in all its glory (only 512 bytes):

:107E0000112484B714BE81FFF0D085E080938100F7
:107E100082E08093C00088E18093C10086E0809377
:107E2000C20080E18093C4008EE0C9D0259A86E02C
:107E300020E33CEF91E0309385002093840096BBD3
:107E4000B09BFECF1D9AA8958150A9F7CC24DD24C4
:107E500088248394B5E0AB2EA1E19A2EF3E0BF2EE7
:107E6000A2D0813461F49FD0082FAFD0023811F036
:107E7000013811F484E001C083E08DD089C08234E0
:107E800011F484E103C0853419F485E0A6D080C0E4
:107E9000853579F488D0E82EFF2485D0082F10E0AE
:107EA000102F00270E291F29000F111F8ED06801E7
:107EB0006FC0863521F484E090D080E0DECF843638
:107EC00009F040C070D06FD0082F6DD080E0C81688
:107ED00080E7D80618F4F601B7BEE895C0E0D1E017
:107EE00062D089930C17E1F7F0E0CF16F0E7DF06D8
:107EF00018F0F601B7BEE89568D007B600FCFDCFD4
:107F0000A601A0E0B1E02C9130E011968C91119780
:107F100090E0982F8827822B932B1296FA010C0160
:107F200087BEE89511244E5F5F4FF1E0A038BF0790
:107F300051F7F601A7BEE89507B600FCFDCF97BE46
:107F4000E89526C08437B1F42ED02DD0F82E2BD052
:107F50003CD0F601EF2C8F010F5F1F4F84911BD097
:107F6000EA94F801C1F70894C11CD11CFA94CF0C13
:107F7000D11C0EC0853739F428D08EE10CD085E9AC
:107F80000AD08FE07ACF813511F488E018D01DD067
:107F900080E101D065CF982F8091C00085FFFCCF94
:107FA0009093C60008958091C00087FFFCCF809118
:107FB000C00084FD01C0A8958091C6000895E0E648
:107FC000F0E098E1908380830895EDDF803219F02E
:107FD00088E0F5DFFFCF84E1DECF1F93182FE3DFCA
:107FE0001150E9F7F2DF1F91089580E0E8DFEE27F6
:047FF000FF270994CA
:027FFE00040479
:0400000300007E007B
:00000001FF

So, a line in the .hex file will start with a colon, then a length, then an address, then a transaction type, then some variable amount of data, and then a sumcheck. These are stored in the file in hexadecimal notation.

The very first thing we can deduce from this is that the file size will be over double the amount of memory that is programmed. For example, to program "AB" in hex, this would take two bytes in the .hex file ("A" followed by "B") but these would be converted into the equivalent one-byte "binary" number, which would end up in the MCU memory.

Then, because of the colon, the transaction type, the line length, and the sumcheck, the file will be somewhat longer still, than double the number of bytes to be programmed into the MCU. That is the file layout overhead.

Are intel hex files converted into a binary format before they are written to Atmega's flash?

1 Answers1