r/PythonLearning • • 3d ago

Showcase byte strings

I found it abit annoying and confusing to represent "some bytes" with ASCII letters. I don't know why python chooses to do so for Unicode characters less than 127. Anyway I wrote my own implementation for UTF-8. Here is the implementation:

string = '😂happy man'
byte_list = []

for char in string:
    binary_string_list = list(f'{ord(char):b}')
    binary_string_list.reverse()
    
    # check length of binary_string
    if len(binary_string_list) <= 7:
        # data represented using a single byte.
        byte_temp = list('0???????')
        byte_temp.reverse()
    
    elif len(binary_string_list) <= 11:
        # data represented using two bytes.
        byte_temp = list('110?????10??????')
        byte_temp.reverse()
        
    elif len(binary_string_list) <= 16:
        # data represented using two bytes.
        byte_temp = list('1110????10??????10??????')
        byte_temp.reverse()
        
    elif len(binary_string_list) <= 21:
        # data represented using two bytes.
        byte_temp = list('11110???10??????10??????10??????')
        byte_temp.reverse()
    
    def imlement_utf(byte_temp, byte_string_list):
        # this function fills the byte templates with actual binary data
        if '?' in byte_temp:
            for bit in byte_string_list:
                byte_temp[byte_temp.index('?')] = bit
                    
            byte_temp.reverse()
            
            for bit in byte_temp:
                if bit == '?':
                    byte_temp[byte_temp.index('?')] = '0'
        
        global byte_list             
        byte_list += byte_temp
    imlement_utf(byte_temp, binary_string_list)

print(f"This is the encoded binary string: {''.join(byte_list)}")
print()
# converting binary string to byte string:
hex_string = ''
f_hex_string = ''
data = ''.join(byte_list)
offsets = zip(range(0, len(data)-7, 8), range(8, len(data)+1, 8))
for x, y in offsets:
    byte = data[x:y]
    # binary to decimal
    byte = int(byte, base=2)
    # decimal to hexadecimal
    byte = f'{byte:x}'
    hex_string += ' '+byte
    # formatted hexadecimal
    f_hex_string += '\\x'+byte
print(f'This is the condensed binary string in hexadecimal: {hex_string.strip()}')
print()

print(f"This is the formarted hexadecimal string (byte string):b'{f_hex_string}'")
print()

print(f"This is the byte string python returns: {bytes.fromhex(hex_string)}")
0 Upvotes

4 comments sorted by

•

u/Sea-Ad7805 3d ago

Run program above by clicking Memory Graph Web Debugger and see its state change step by step.

1

u/mc_pm 2d ago

What exactly is the problem you think you're solving with this?

2

u/osamas_den 2d ago

none,,,just wanted to peek at the bytes

0

u/mc_pm 2d ago

You might want to look at using encoding/decoding in Python. It will take the string from UTF-8, turn it into bytes, then turn that back into whatever encoding you want.