How do you convert a byte array to a hexadecimal string, and vice versa? – Dev

The best answers to the question “How do you convert a byte array to a hexadecimal string, and vice versa?” in the category Dev.

QUESTION:

How can you convert a byte array to a hexadecimal string, and vice versa?

ANSWER:

Performance Analysis

Note: new leader as of 2015-08-20.

I ran each of the various conversion methods through some crude Stopwatch performance testing, a run with a random sentence (n=61, 1000 iterations) and a run with a Project Gutenburg text (n=1,238,957, 150 iterations). Here are the results, roughly from fastest to slowest. All measurements are in ticks (10,000 ticks = 1 ms) and all relative notes are compared to the [slowest] StringBuilder implementation. For the code used, see below or the test framework repo where I now maintain the code for running this.

Disclaimer

WARNING: Do not rely on these stats for anything concrete; they are simply a sample run of sample data. If you really need top-notch performance, please test these methods in an environment representative of your production needs with data representative of what you will use.

Results

  • Lookup by byte unsafe (via CodesInChaos) (added to test repo by airbreather)
    • Text: 4,727.85 (105.2X)
    • Sentence: 0.28 (99.7X)
  • Lookup by byte (via CodesInChaos)
    • Text: 10,853.96 (45.8X faster)
    • Sentence: 0.65 (42.7X faster)
  • Byte Manipulation 2 (via CodesInChaos)
    • Text: 12,967.69 (38.4X faster)
    • Sentence: 0.73 (37.9X faster)
  • Byte Manipulation (via Waleed Eissa)
    • Text: 16,856.64 (29.5X faster)
    • Sentence: 0.70 (39.5X faster)
  • Lookup/Shift (via Nathan Moinvaziri)
    • Text: 23,201.23 (21.4X faster)
    • Sentence: 1.24 (22.3X faster)
  • Lookup by nibble (via Brian Lambert)
    • Text: 23,879.41 (20.8X faster)
    • Sentence: 1.15 (23.9X faster)
  • BitConverter (via Tomalak)
    • Text: 113,269.34 (4.4X faster)
    • Sentence: 9.98 (2.8X faster)
  • {SoapHexBinary}.ToString (via Mykroft)
    • Text: 178,601.39 (2.8X faster)
    • Sentence: 10.68 (2.6X faster)
  • {byte}.ToString("X2") (using foreach) (derived from Will Dean’s answer)
    • Text: 308,805.38 (2.4X faster)
    • Sentence: 16.89 (2.4X faster)
  • {byte}.ToString("X2") (using {IEnumerable}.Aggregate, requires System.Linq) (via Mark)
    • Text: 352,828.20 (2.1X faster)
    • Sentence: 16.87 (2.4X faster)
  • Array.ConvertAll (using string.Join) (via Will Dean)
    • Text: 675,451.57 (1.1X faster)
    • Sentence: 17.95 (2.2X faster)
  • Array.ConvertAll (using string.Concat, requires .NET 4.0) (via Will Dean)
    • Text: 752,078.70 (1.0X faster)
    • Sentence: 18.28 (2.2X faster)
  • {StringBuilder}.AppendFormat (using foreach) (via Tomalak)
    • Text: 672,115.77 (1.1X faster)
    • Sentence: 36.82 (1.1X faster)
  • {StringBuilder}.AppendFormat (using {IEnumerable}.Aggregate, requires System.Linq) (derived from Tomalak’s answer)
    • Text: 718,380.63 (1.0X faster)
    • Sentence: 39.71 (1.0X faster)

Lookup tables have taken the lead over byte manipulation. Basically, there is some form of precomputing what any given nibble or byte will be in hex. Then, as you rip through the data, you simply look up the next portion to see what hex string it would be. That value is then added to the resulting string output in some fashion. For a long time byte manipulation, potentially harder to read by some developers, was the top-performing approach.

Your best bet is still going to be finding some representative data and trying it out in a production-like environment. If you have different memory constraints, you may prefer a method with fewer allocations to one that would be faster but consume more memory.

Testing Code

Feel free to play with the testing code I used. A version is included here but feel free to clone the repo and add your own methods. Please submit a pull request if you find anything interesting or want to help improve the testing framework it uses.

  1. Add the new static method (Func<byte[], string>) to /Tests/ConvertByteArrayToHexString/Test.cs.
  2. Add that method’s name to the TestCandidates return value in that same class.
  3. Make sure you are running the input version you want, sentence or text, by toggling the comments in GenerateTestInput in that same class.
  4. Hit F5 and wait for the output (an HTML dump is also generated in the /bin folder).
static string ByteArrayToHexStringViaStringJoinArrayConvertAll(byte[] bytes) {
    return string.Join(string.Empty, Array.ConvertAll(bytes, b => b.ToString("X2")));
}
static string ByteArrayToHexStringViaStringConcatArrayConvertAll(byte[] bytes) {
    return string.Concat(Array.ConvertAll(bytes, b => b.ToString("X2")));
}
static string ByteArrayToHexStringViaBitConverter(byte[] bytes) {
    string hex = BitConverter.ToString(bytes);
    return hex.Replace("-", "");
}
static string ByteArrayToHexStringViaStringBuilderAggregateByteToString(byte[] bytes) {
    return bytes.Aggregate(new StringBuilder(bytes.Length * 2), (sb, b) => sb.Append(b.ToString("X2"))).ToString();
}
static string ByteArrayToHexStringViaStringBuilderForEachByteToString(byte[] bytes) {
    StringBuilder hex = new StringBuilder(bytes.Length * 2);
    foreach (byte b in bytes)
        hex.Append(b.ToString("X2"));
    return hex.ToString();
}
static string ByteArrayToHexStringViaStringBuilderAggregateAppendFormat(byte[] bytes) {
    return bytes.Aggregate(new StringBuilder(bytes.Length * 2), (sb, b) => sb.AppendFormat("{0:X2}", b)).ToString();
}
static string ByteArrayToHexStringViaStringBuilderForEachAppendFormat(byte[] bytes) {
    StringBuilder hex = new StringBuilder(bytes.Length * 2);
    foreach (byte b in bytes)
        hex.AppendFormat("{0:X2}", b);
    return hex.ToString();
}
static string ByteArrayToHexViaByteManipulation(byte[] bytes) {
    char[] c = new char[bytes.Length * 2];
    byte b;
    for (int i = 0; i < bytes.Length; i++) {
        b = ((byte)(bytes[i] >> 4));
        c[i * 2] = (char)(b > 9 ? b + 0x37 : b + 0x30);
        b = ((byte)(bytes[i] & 0xF));
        c[i * 2 + 1] = (char)(b > 9 ? b + 0x37 : b + 0x30);
    }
    return new string(c);
}
static string ByteArrayToHexViaByteManipulation2(byte[] bytes) {
    char[] c = new char[bytes.Length * 2];
    int b;
    for (int i = 0; i < bytes.Length; i++) {
        b = bytes[i] >> 4;
        c[i * 2] = (char)(55 + b + (((b - 10) >> 31) & -7));
        b = bytes[i] & 0xF;
        c[i * 2 + 1] = (char)(55 + b + (((b - 10) >> 31) & -7));
    }
    return new string(c);
}
static string ByteArrayToHexViaSoapHexBinary(byte[] bytes) {
    SoapHexBinary soapHexBinary = new SoapHexBinary(bytes);
    return soapHexBinary.ToString();
}
static string ByteArrayToHexViaLookupAndShift(byte[] bytes) {
    StringBuilder result = new StringBuilder(bytes.Length * 2);
    string hexAlphabet = "0123456789ABCDEF";
    foreach (byte b in bytes) {
        result.Append(hexAlphabet[(int)(b >> 4)]);
        result.Append(hexAlphabet[(int)(b & 0xF)]);
    }
    return result.ToString();
}
static readonly uint* _lookup32UnsafeP = (uint*)GCHandle.Alloc(_Lookup32, GCHandleType.Pinned).AddrOfPinnedObject();
static string ByteArrayToHexViaLookup32UnsafeDirect(byte[] bytes) {
    var lookupP = _lookup32UnsafeP;
    var result = new string((char)0, bytes.Length * 2);
    fixed (byte* bytesP = bytes)
    fixed (char* resultP = result) {
        uint* resultP2 = (uint*)resultP;
        for (int i = 0; i < bytes.Length; i++) {
            resultP2[i] = lookupP[bytesP[i]];
        }
    }
    return result;
}
static uint[] _Lookup32 = Enumerable.Range(0, 255).Select(i => {
    string s = i.ToString("X2");
    return ((uint)s[0]) + ((uint)s[1] << 16);
}).ToArray();
static string ByteArrayToHexViaLookupPerByte(byte[] bytes) {
    var result = new char[bytes.Length * 2];
    for (int i = 0; i < bytes.Length; i++)
    {
        var val = _Lookup32[bytes[i]];
        result[2*i] = (char)val;
        result[2*i + 1] = (char) (val >> 16);
    }
    return new string(result);
}
static string ByteArrayToHexViaLookup(byte[] bytes) {
    string[] hexStringTable = new string[] {
        "00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "0A", "0B", "0C", "0D", "0E", "0F",
        "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "1A", "1B", "1C", "1D", "1E", "1F",
        "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "2A", "2B", "2C", "2D", "2E", "2F",
        "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "3A", "3B", "3C", "3D", "3E", "3F",
        "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "4A", "4B", "4C", "4D", "4E", "4F",
        "50", "51", "52", "53", "54", "55", "56", "57", "58", "59", "5A", "5B", "5C", "5D", "5E", "5F",
        "60", "61", "62", "63", "64", "65", "66", "67", "68", "69", "6A", "6B", "6C", "6D", "6E", "6F",
        "70", "71", "72", "73", "74", "75", "76", "77", "78", "79", "7A", "7B", "7C", "7D", "7E", "7F",
        "80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "8A", "8B", "8C", "8D", "8E", "8F",
        "90", "91", "92", "93", "94", "95", "96", "97", "98", "99", "9A", "9B", "9C", "9D", "9E", "9F",
        "A0", "A1", "A2", "A3", "A4", "A5", "A6", "A7", "A8", "A9", "AA", "AB", "AC", "AD", "AE", "AF",
        "B0", "B1", "B2", "B3", "B4", "B5", "B6", "B7", "B8", "B9", "BA", "BB", "BC", "BD", "BE", "BF",
        "C0", "C1", "C2", "C3", "C4", "C5", "C6", "C7", "C8", "C9", "CA", "CB", "CC", "CD", "CE", "CF",
        "D0", "D1", "D2", "D3", "D4", "D5", "D6", "D7", "D8", "D9", "DA", "DB", "DC", "DD", "DE", "DF",
        "E0", "E1", "E2", "E3", "E4", "E5", "E6", "E7", "E8", "E9", "EA", "EB", "EC", "ED", "EE", "EF",
        "F0", "F1", "F2", "F3", "F4", "F5", "F6", "F7", "F8", "F9", "FA", "FB", "FC", "FD", "FE", "FF",
    };
    StringBuilder result = new StringBuilder(bytes.Length * 2);
    foreach (byte b in bytes) {
        result.Append(hexStringTable[b]);
    }
    return result.ToString();
}

Update (2010-01-13)

Added Waleed’s answer to analysis. Quite fast.

Update (2011-10-05)

Added string.Concat Array.ConvertAll variant for completeness (requires .NET 4.0). On par with string.Join version.

Update (2012-02-05)

Test repo includes more variants such as StringBuilder.Append(b.ToString("X2")). None upset the results any. foreach is faster than {IEnumerable}.Aggregate, for instance, but BitConverter still wins.

Update (2012-04-03)

Added Mykroft’s SoapHexBinary answer to analysis, which took over third place.

Update (2013-01-15)

Added CodesInChaos’s byte manipulation answer, which took over first place (by a large margin on large blocks of text).

Update (2013-05-23)

Added Nathan Moinvaziri’s lookup answer and the variant from Brian Lambert’s blog. Both rather fast, but not taking the lead on the test machine I used (AMD Phenom 9750).

Update (2014-07-31)

Added @CodesInChaos’s new byte-based lookup answer. It appears to have taken the lead on both the sentence tests and the full-text tests.

Update (2015-08-20)

Added airbreather’s optimizations and unsafe variant to this answer’s repo. If you want to play in the unsafe game, you can get some huge performance gains over any of the prior top winners on both short strings and large texts.

ANSWER:

You can use Convert.ToHexString starting with .NET 5.
There’s also a method for the reverse operation: Convert.FromHexString.


For older versions of .NET you can either use:

public static string ByteArrayToString(byte[] ba)
{
  StringBuilder hex = new StringBuilder(ba.Length * 2);
  foreach (byte b in ba)
    hex.AppendFormat("{0:x2}", b);
  return hex.ToString();
}

or:

public static string ByteArrayToString(byte[] ba)
{
  return BitConverter.ToString(ba).Replace("-","");
}

There are even more variants of doing it, for example here.

The reverse conversion would go like this:

public static byte[] StringToByteArray(String hex)
{
  int NumberChars = hex.Length;
  byte[] bytes = new byte[NumberChars / 2];
  for (int i = 0; i < NumberChars; i += 2)
    bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
  return bytes;
}

Using Substring is the best option in combination with Convert.ToByte. See this answer for more information. If you need better performance, you must avoid Convert.ToByte before you can drop SubString.

ANSWER:

When writing crypto code it’s common to avoid data dependent branches and table lookups to ensure the runtime doesn’t depend on the data, since data dependent timing can lead to side-channel attacks.

It’s also pretty fast.

static string ByteToHexBitFiddle(byte[] bytes)
{
    char[] c = new char[bytes.Length * 2];
    int b;
    for (int i = 0; i < bytes.Length; i++) {
        b = bytes[i] >> 4;
        c[i * 2] = (char)(55 + b + (((b-10)>>31)&-7));
        b = bytes[i] & 0xF;
        c[i * 2 + 1] = (char)(55 + b + (((b-10)>>31)&-7));
    }
    return new string(c);
}

Ph’nglui mglw’nafh Cthulhu R’lyeh wgah’nagl fhtagn


Abandon all hope, ye who enter here

An explanation of the weird bit fiddling:

  1. bytes[i] >> 4 extracts the high nibble of a byte
    bytes[i] & 0xF extracts the low nibble of a byte
  2. b - 10
    is < 0 for values b < 10, which will become a decimal digit
    is >= 0 for values b > 10, which will become a letter from A to F.
  3. Using i >> 31 on a signed 32 bit integer extracts the sign, thanks to sign extension.
    It will be -1 for i < 0 and 0 for i >= 0.
  4. Combining 2) and 3), shows that (b-10)>>31 will be 0 for letters and -1 for digits.
  5. Looking at the case for letters, the last summand becomes 0, and b is in the range 10 to 15. We want to map it to A(65) to F(70), which implies adding 55 ('A'-10).
  6. Looking at the case for digits, we want to adapt the last summand so it maps b from the range 0 to 9 to the range 0(48) to 9(57). This means it needs to become -7 ('0' - 55).
    Now we could just multiply with 7. But since -1 is represented by all bits being 1, we can instead use & -7 since (0 & -7) == 0 and (-1 & -7) == -7.

Some further considerations:

  • I didn’t use a second loop variable to index into c, since measurement shows that calculating it from i is cheaper.
  • Using exactly i < bytes.Length as upper bound of the loop allows the JITter to eliminate bounds checks on bytes[i], so I chose that variant.
  • Making b an int allows unnecessary conversions from and to byte.

ANSWER:

There’s a class called SoapHexBinary that does exactly what you want.

using System.Runtime.Remoting.Metadata.W3cXsd2001;

public static byte[] GetStringToBytes(string value)
{
    SoapHexBinary shb = SoapHexBinary.Parse(value);
    return shb.Value;
}

public static string GetBytesToString(byte[] value)
{
    SoapHexBinary shb = new SoapHexBinary(value);
    return shb.ToString();
}