Submitted by matt_eaton
on Fri, 11/24/2017 - 11:09 PM
Recently I was in a situation where I needed to convert hexadecimal data into ASCII text data and I had to pause because I needed to remember how this was done in a low level environment and it was escaping me off the top of my head. I was working close to the metal in a Darwin environment so I had C at my disposal, but I could not for the life of me remember the conversion from hexadecimal to ASCII for a stream of data like 0x6a, 0xdc. So that is the goal for this post, to describe my thought process on how I worked this conversion out and to validate this conversion was correct by writing a test to validate that the conversion process I was using was accurate from an ASCII and from a mathematical perspective.
NOTE: The following blog post assumes a clang / gcc Darwin environment with C and Python available, but there is nothing out of the ordinary about this code in this post so the following code should working in Linux too.
Solving the Problem
In researching the problem I found out that C has a very nice solution to this problem and that was to use sprintf to convert the buffer of hexadecimal data to a char array using the %x formatter tag to signify that we were in fact working with hexadecimal data. This solution then takes the hexadecimal data and writes it to a char array that you have provided to be used in ASCII form. Pretty interesting, and pretty straight forward once I understood the principles of the solution. One thing that had me curious though is that I was testing this solution with data of a specific format in a specific size; what would happen if I all of a sudden had hexadecimal data that took all various shapes and sizes. This seemed like a more real world scenario and something that I needed to test out before I felt better about my solution.
// Value between 0 and 255 unsigned char hexData[SMALL_BUFFER_SIZE] = {0x6a, 0xdc}; char hexSwapArray[4]; for (int i = 0; i < SMALL_BUFFER_SIZE; i += 1) { // hexSwapArray contains the original characters and hexData has the // current set being formatted as lowercase hexidecimal sprintf(hexSwapArray, "%x", hexData[i]); printf("char string: %s\n", hexSwapArray); }
Testing our Solution
To test my solution of using sprintf, I needed a way to validate that the output I was getting from sprintf was accurate. One way I thought to do this was to manually create an algorithm that converted the decimal form of the hex data into ASCII data so I could then compare the output from sprintf to my hand converted text to see if I had a match. This match would validate that my solution was correct and allow me to continue on with my project with tested code.
To create this test I created a C module using an includable header file. That way if I needed this test to be included anywhere else in the project, it would be portable enough and could be with one single line of code.
To test my sprintf code I wrote a loop that took a buffer of unsigned integer hexadecimal data and looped through it and sent each value to a testing function as an integer decimal value. From here I created three character buffers, one for the reverseBuffer, that we will use to reverse engineer our decimal value, one for the swapBuffer that we will use to swap our hex data into ASCII, and the last for the compareBuffer to compare the reverse engineered value to our sprintf swap buffer to get the results of our tests. The decimal value passed in to the test function is then iterated through with a while loop and the hex characters are generated from the decimal remainder of being divided by 16 with each iteration. Each iteration is added to our reverseBuffer in the while loop and the decimal value is divided lower and lower. The reason this buffer is called a reverseBuffer is because this buffer is actually the ASCII value that is built in reverse, so the for loop underneath the while loop actually then takes the reverseBuffer and builds the compareBuffer. This compare buffer is then used to be validate against our swap buffer to give us the results of our test. The results then validates the usage of sprintf!
The test to validate sprintf is not the most efficient use of memory in the context of C, because 3 buffers are declared with a fixed set of memory, but if were used in a program this may be better served to be dynamically allocated using calloc instead.
/** * Our test function manually converts the decimal represented decimalHexValue * to a hex value by hand. * * The hex value is then compared against the output of sprintf to check accuracy * and to check if we can assert accuracy on sprintf for converting hex values. */ int test_sprintf_hex_data(unsigned int decimalHexValue, int size) { // This implementation is fairly wasteful in terms of memory // The three char arrays represent an optomistic chunk of memory to prove our test // Using an assigned char limit could be dangerous in this situation // It might be better to look into dynamically allocation memory using calloc in a running app char hexChars[16] = "0123456789abcdef"; unsigned int decimalValue = decimalHexValue; char reverseBuffer[10], swapBuffer[10], compareBuffer[10]; int whileFlag = 1, index = 0, reverseIndex = 0; // Zero out the reverseBuffer and compareBuffer for potential memory issues for (int j = 0; j < 10; j += 1) { reverseBuffer[j] = 0; compareBuffer[j] = 0; } while(whileFlag) { if (decimalValue > 0) { reverseBuffer[index] = hexChars[decimalValue % 16]; index += 1; decimalValue = decimalValue / 16; } else { whileFlag = 0; } } if (index > 0) { for (int e = (index-1); e >= 0; e -= 1) { compareBuffer[reverseIndex] = reverseBuffer[e]; reverseIndex += 1; } } else { compareBuffer[reverseIndex] = '0'; } // Print out the swap and compare buffers sprintf(swapBuffer, "%x", decimalHexValue); printf("swap buffer : %s\n", swapBuffer); printf("compare buffer: %s\n", compareBuffer); return strcmp(swapBuffer, compareBuffer); }
Building our Program
To build this program and testing module I wrote a small Python utility that compiles and builds everything for you using gcc. This is optional, and there certainly are better ways to do this, but I thought it would be easy to use with one command via the command line. This utility cleans up all of the object files, finds all of the .c files and compiles them into .o files and then links them together creating a main binary. This utility does not introspect the actual text of your program code for headers to compile but just directories instead.
#!/usr/bin/env python # -*- coding: utf-8 -*- # # Import needed stdlib and other dependencies # from __future__ import print_function import os, sys, time, subprocess, shutil, datetime # Iterate through files in this directory and remove all old .o files for file_item in os.listdir('.'): if file_item[-2:] == '.o': print("Found .o file: " + file_item) os.remove(file_item) # Iterate through files in this directory and compile all .c files to .o files for file_item in os.listdir('.'): if file_item[-2:] == '.c': print("Found .c file: " + file_item) compile_command = "gcc -c " + file_item + " -o " + file_item[:(len(file_item) -2)] + ".o" os.system(compile_command) # Iterate through files in this directory and capture them in a list object_files = [] for file_item in os.listdir('.'): if file_item[-2:] == '.o': object_files.append(file_item) # Iterate through files in this directory and link them together to form a binary final_command = "gcc " for object_file in object_files: final_command += object_file + " " final_command += " -o main" # Build the binary os.system(final_command) # Executes the binary print("PROGRAM OUT: ") os.system("./main")
Thank you for reading, the code for this post can be found on my Github page here, and if you have any questions, comment, or concerns, I would love to hear from you!
Member for
3 years 9 monthsLong time mobile team lead with a love for network engineering, security, IoT, oss, writing, wireless, and mobile. Avid runner and determined health nut living in the greater Chicagoland area.