[drxddock]

The 0's and 1's of Computer Science
Ever seen those fancy 0's and 1's on a stock photo image of some hacker at a computer? Or maybe you've seen the classic green-on-black 01010101's on the Matrix movies. Well, have you ever wondered what they actually mean..?

So what's (hexa)decimal?
In order for you to begin understanding binary and how it works, you must first understand how decimal or hexadecimal works. We'll start with decimal:
In decimal, every character or signal is represented by a number, using a character set of 0-9. For example, the letter 'A' (note: capital.) is represented by the decimal value 65. the letter 'B' is represented by decimal value 66. So on, so forth. All printable characters can be represented by decimal, and the printable characters start at decimal value 32, which represents space. I'll link a chart showing the decimal and hexadecimal values of each character at the end of this post. But what about the decimal values before 32?

Non-printable Characters
Metadata: It's beautiful! Metadata is data about the data. The file size of a document is a great example of metadata. It's data giving information (file size) about more data (the document). Some of the "characters" of a decimal value before 32 are metadata. For example, decimal value 2 means "start of text." Decimal value 3 means "end of text." Decimal value 4 means "end of transmission." This is great data that will help a receiving computer better understand the data that is sent. For example, a message could be sent that looks like this: "[decimal value 2] Hey there! [decimal value 3] [decimal value 4]" This message could then be received and understood as: "(beginning of text) Hey there! (end of text) (end of transmission)" to help the receiver better understand and structure the message.

But what about the other non-printable characters?
Well, most or all of them are CPU instruction. A CPU is the Central Processing Unit of a computer or device. It's a chip that receives instructions or tasks on what to do (called CPU instruction, Processor instruction, bytecode, or shellcode if used to spawn an interactive shell like bash), and then proceeds to execute that task and return an output. For example, decimal value 144 stands for bytecode that means "No Operation", or can be shortened to NOP. When a processor (CPU) reaches this instruction, it will move on to the next instruction byte.

In summation, Non-Printable Characters are characters to be read by processors to be used for instruction of execution or can be used as metadata to help structure receiving or sending data.

Great! I understand decimal. Now what's hexadecimal..?
Well, as you know, decimal is another way of representing printable characters, instruction, or metadata in a byte. You'll be happy to hear that hexadecimal does the exact same thing! The only difference is that hexadecimal will count up using letters as well as numbers. Hexadecimal has a counting character set of 0-9 and a-f. But why is this better..? Well, each byte, whether representing a printable or non-printable character can be used in a hexadecimal value in exactly two characters. Always. For example, where decimal will count up in the order of 01, 02, 03, [...], 09, 10, 11, 12, hexadecimal will count up in the order of 01, 02, 03, [...], 09, 0a, 0b, 0c, 0d, 0e, 0f, 10, 11, 12. This is called a base-16 number system. It means that the tens place goes up once for every sixteen counts. Confusing? Let me help by giving a count to 20 in both representations.

Decimal: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20.
Hexadecimal: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0a, 0b, 0c, 0d, 0e, 0f, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1a, 1b, 1c, 1d, 1e, 1f, 20.

So, while decimal 10 will represent [NEWLINE] (the non-printable instruction for printing a new line, like hitting the "return" key on your keyboard), it could be represented in hexadecimal as 0a.

What does that have to do with binary?
All those fancy 0's and 1's you see represent the values that you just learned about! Now, I taught you about both decimal and hexadecimal. I did this because there are two ways to interpret or translate the binary: into decimal values, and into hexadecimal values. Personally, I like hexadecimal more because I've had better experience with using it to represent non-printable characters in the past, and it fits better in a binary translation. Although, if you wish to learn how to translate binary to decimal instead of hexadecimal, feel free to skip down to the "Translating Binary to Decimal Values" header later in this post.

Translating Binary to Hexadecimal Values
First things first: Each byte is represented in binary by exactly eight 0's or 1's. It's like the same thing of how each byte is represented in hexadecimal by 2 characters of 0-9 or a-f. 8 divided by 2 = 4.. So, there are exactly four 0's or 1's for each hexadecimal character. But how does this work..? Well, I'll give you an example:

10010000

Obviously you shouldn't know what this means yet. But let's dive right into it:
As previously stated, there are four 0's or 1's for a hexadecimal character. Let's go ahead and focus on the first four: '1001'. There are four "slots" for 0's or 1's to go into. You can see the first "slot" is stated as 1, the second and third "slots" are stated as 0, and the fourth "slot" is stated as a 1. Now, it might look a little complicated at first, but it's only just simple addition. As you may know, 1 often stands for 'true', and 0 often stands for 'false'. Each of these slots stands for a certain numerical value. The first slot stands for 8. The second slot stands for 4. The third slot stands for 2, and the fourth slot stands for 1. So, as you can see in the example of '1001', the first slot, which stands for 8, is true and so is the fourth slot, which stands for 1. That gives us an 8 and a 1, so we add those numbers. From there, we get 9. So, we can determine that 1001 in binary stands for hexadecimal character 9. But we only have half the byte.

Well, we do the same thing for the last four characters, too. Using that same logic, binary '0000' stands for hexadecimal '0'. So, binary '10010000' must stand for hexadecimal '90', which is the non-printable instruction for NOP.

But we're missing something huge. Hexadecimal doesn't only use numbers, it also uses a-f! How are we supposed to translate binary numbers into a hexadecimal letter?! Well, it's quite simple actually. If you get the numerical value for 10, then the hexadecimal value is 'a'. If you get 11, hexadecimal value is 'b'. 12, 'c'. 13, 'd', 14, 'e'. And finally, 15, 'f'. So, the binary translation for hexadecimal value '0a' could be represented as '00001010', because '0000' translates to 0, and '1010' gives us a numerical value of 10 (first slot [8] + third slot [2]), which stands for hexadecimal value 'a'.

Translating Binary to Decimal Values
First things first: Each byte is represented in binary by exactly eight 0's or 1's. Just like in hexadecimal, there are "slots" that stand for different numerical values. Instead of splitting the byte into two different sections, four slots for each, we take on the byte as a whole. The first slot stands for the numerical value of 128, and each slot stands for half of the last. To sum this up, the first slot stands for 128, the second 64, the third 32, the fourth 16, the fifth 8, the sixth 4, the seventh 2, and the eighth 1. As previously mentioned in the last section, each slot can either have a '0' or a '1' in it, the '0' meaning 'false' and the '1' meaning 'true'. We will assign a numerical value to the byte based on which slots have 1's in them. For example, if the first slot contains a 1, you should add 128 to the numerical value. If the second slot contains a 1, you should add 64 to the numerical value. So on, so forth. The numerical value that you receive is a decimal representation of a byte.

Example:
10111011

The first, third, fourth, fifth, seventh, and eighth slots contain a 1. Therefore, we should add their respective values and we will get a decimal output. We already know the values of the slots, so we should go ahead and add those values. Let's do that.

128 (first slot) + 32 (third slot) + 16 (fourth) + 8 (fifth) + 2 (seventh) + 1 (eighth) = 187.

Decimal 187 is an extended ascii printable character. It looks like a corner that's used in ascii art for designing.

Summation & Extras
In conclusion, a byte is represented in binary as eight 0's or 1's, and in hexadecimal as two characters 0-9 or a-f. There are two ways to translate binary into characters: translating it to decimal or to hexadecimal. More info can be found up above.

Helpful Links
A great lookup table for decimal and hexadecimal values can be found here.

Challenge!
Convert this to decimal or hexadecimal based on your personal choice. Then, convert that to the printable characters that they represent. No online translators! [That's cheating ]

Code:

01000011011011110110111001100111011100100110000101110100011101010110110001100001011101000110100101101111011011100111001100100001

[Note] Don't forget to hold the answer in a spoiler for those who wish to participate on their own.
[Note] Dear mods: This is my first thread. Not sure if I'm in the right sub. Feel free to move if necessary.

[Synthx]

I'm glad to see that you know how to write high quality threads! Thanks for this tutorial, nice and easy to read.

[Blink]

Nicely written.
I got used to binary long before I did hex, so you don't really need to be good with hex before binary...

Your thing with non-printing characters doesn't exactly sound right. CPU instructions have nothing to do with printing or non-printing characters, and generally some CPU instructions would also be printing characters if rendered as ASCII, USF-8, EBCDIC, or another character encoding.

You didn't really go far into character encodings, so I'll do that now. If I say 'you', I'm talking to everyone reading because I'll write this like a tutorial:

In order for your computer to display characters, they need to be stored somehow. Computers generally store data in ways that can be accessed as HIGH and LOW values, with HIGH being a specific voltage level, and LOW being another. Since we have two values, we can store numbers as binary values, with HIGH being 1, and LOW being 0.
Now, how do we store letters? Well, that's actually fairly simple. We encode those characters as numbers. The easiest way for you to understand what I'm talking about is for me to give an example, so I'll show you a very simple character encoding:

Code:

A - 1 B - 2 C - 3 D - 4 E - 5 F - 6 G - 7 H - 8 I - 9 J - 10 K - 11 L - 12 M - 13 N - 14 O - 15 P - 16 Q - 17 R - 18 S - 19 T - 20 U - 21 V - 22 W - 23 X - 24 Y - 25 Z - 26 SPACE - 27

Alright, now, to store the message of "HELLO", we need to simply convert each letter to a number, but since computers generally use two values, we'll store it as a binary value:

Code:

H -> 8 -> 1000 E -> 5 -> 101 L -> 12 -> 1100 O -> 15 -> 1111 END -> 0 HELLO: 1000 101 1100 1100 1111 0

Notice how we have an END character? That's because the computer needs a way to know where the message ends, otherwise it will start printing everything that's in memory. Generally we use 0, this is called a null byte.
Now, the computer can't just store the message as shown above, since computers generally deal with binary numbers of a specific length. 32 bit computers generally deal with 32 bit long numbers, 8 bit computers deal with 8 bit ones, etc. How do we make these numbers longer? We just add 0s on the left side, since it still equals the same number. A byte really means the smallest amount of bits a computer can work with, but we generally just mean '8 bits' when we say byte. Stored as 8 bit values, our message looks like this:

Code:

00001000 00000101 00001100 00001100 00001111 00000000

Now, there aren't really any gaps between the numbers, there's no reason for that if the computer knows that all numbers are 8 bits long:

Code:

000010000000010100001100000011000000111100000000

However, we generally use spaces when looking at binary numbers, because it's easier for us to see it that way.

Great, so now we have "HELLO" stored, but we want more than just capital letters right? So, let's make a new encoding, with more symbols:

Code:

END - 0 A - 1 B - 2 C - 3 D - 4 E - 5 ... Z - 26 SPACE - 27 / - 28 - 29 (a dash) + - 30 a - 31 b - 32 c - 33 d - 34 e - 35 f - 36 ... z - 56 > - 57 < - 58 = - 59 0 - 60 (the number 0, which will display 0) 1 - 61 2 - 62 3 - 63 ... 9 - 69 ( - 70 ) - 71 ! - 72 $ - 73 [ - 74 ] - 75 . - 76 , - 77 ; - 78 : - 79 " - 80

Alright, that seems like a good amount of characters. We need to encode numbers too, or when a computer encounters a 00000101 (5 in binary), it won't know if it should display an 'E' or a '5', so we encode them too.

Now, we can't just have everyone storing characters however they want, or we'll have a million different encodings, so we have some standardized ones. The two most common are ASCII and UTF-8. UTF-8 takes 4 bytes (so 4 x 8 bits) to store one character, as it has a lot of characters available. The first few UTF-8 characters are the same as ASCII, as it's just a superset of ASCII.
Here's an ASCII table:

Code:

-------------------------------------------------------------------------- 0 nul 1 soh 2 stx 3 etx 4 eot 5 enq 6 ack 7 bel 8 bs 9 ht 10 nl 11 vt 12 np 13 cr 14 so 15 si 16 dle 17 dc1 18 dc2 19 dc3 20 dc4 21 nak 22 syn 23 etb 24 can 25 em 26 sub 27 esc 28 fs 29 gs 30 rs 31 us 32 sp 33 ! 34 " 35 # 36 $ 37 % 38 & 39 ' 40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 / 48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7 56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ? 64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G 72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O 80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W 88 X 89 Y 90 Z 91 [ 92 \ 93 ] 94 ^ 95 _ 96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g 104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o 112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w 120 x 121 y 122 z 123 { 124 | 125 } 126 ~ 127 del

The first few are non-printing, as they do something different. However, they are not just CPU instructions, they're generally processing information about the text. Or, they just do something weird like 7, which makes a bell noise (yes, noise, nothing to do with graphics).
UTF-8 is what's taking over the computing world, and has been doing so since Plan9 first supported it. (Plan9 is an old OS from Bell Labs, it was one of the early OSs to support UTF-8).
However, there isn't just UTF-8 and ASCII, we also have other ones like EBCDIC, which was used in older IBM machines.
Now you guys should have a good understanding of how character encoding works, and you can try to decode his message to ASCII, or another one for fun.

[mothered]

I remember delving Into this around 20 years ago.

An In depth, yet very well documented and easy to understand guide.
Well done.