Language parsing - Printable Version

Language parsing - Printable Version

+- Sinisterly (https://sinister.ly)
+-- Forum: Coding (https://sinister.ly/Forum-Coding)
+--- Forum: Coding (https://sinister.ly/Forum-Coding--71)
+--- Thread: Language parsing (/Thread-Language-parsing)

Pages: 1 2

Language parsing - phyrrus9 - 01-01-2017

So, I got bored the other day and decided to write a descent parser for a made up programming language. The language structure is as follows:

variables:
variables are dynamic and are created on initialization, the parser will throw an error if they are used before then. Variable names have no rules except that they MUST be 31 characters long or less, and must not include the characters +-/*$= (it will let you assign these, but they won't be usable)
Assigning a variable can follow any of these formats:

Code:
name = <constant>
name = <variable>
name = <variable> +-*/% <variable>
name = <variable> +-*/% <constant>

the order (var + 5 or 5 + var) is not important

Arithmatic operations
add (+)
subtract (-)
multiply (*)
divide (/)
modulo (%)

Special features:

printing variables to screen
there exists two specials for printing variables:

Code:
.print

prints all variables in list to screen

Code:
.print(name)

prints the variable "name" to screen

deleting variables

Code:
.delete(name)

calling functions
There are two ways to call functions, always and conditionally.

Code:
.call(procname)

will load the file procname, copy passed variables and execute until return

Code:
.ifcall(procname, condition)

will load procname, copy passed variables, and execute until return if and only if condition evaluates to zero

Returning from a procedure

Code:
.done

When executed in the main context will end the program, when executed in the context of a subprocedure will copy back variables, clean up the list, and return

Reading input

Code:
.input(name)

will read a value from user and store it in variable named name

Printing output

Code:
.echo(text to output)

this is self explanatory.

Notes on calling functions:
variables are passed by value, and utilize a prefix. Only variables prefixed with the function name and a dot will be passed in or out. Example:

Code:
a = 5
i = 7
m.a = a
m.i = i
m.i = m.i + 1
.call(m)

function m will only have access to the variables m.a and m,i with values 5 and 8 respectively.

Sample program:
sample

Code:
.echo(Do you want to execute program? 1/0)
.input(firstexec)
.ifcall(stuff, firstexec - 1)
.echo(Running proc stuff, this may be the second time)
.call(stuff)
.done

stuff

Code:
.echo(Enter first number)
.input(m.a)
.echo(Enter second number)
.input(m.b)
.call(m)
add = m.add
sub = m.sub
mul = m.mul
div = m.div
mod = m.mod
.print(add)
.print(sub)
.print(mul)
.print(div)
.print(mod)
.done

mathops

Code:
a = m.a
b = m.b
m.add = a + b
m.sub = a - b
m.mul = a * b
m.div = a / b
m.mod = a % b
.done

You can find the source code for this here. If you are interested, I may write a tutorial on language parsing and creating your own language.

PLEASE give feedback!

RE: Language parsing - pvnk - 01-02-2017

This is a great contribution to this forum! Thank you!

Biggrin

RE: Language parsing - Nil - 01-02-2017

This is well beyond my purview. You're a very talented individual though.

RE: Language parsing - pvnk - 01-02-2017

(01-02-2017, 01:14 AM)God Wrote: This is well beyond my purview. You're a very talented individual though.

Why thank you! You're much too kind. Smile

RE: Language parsing - Blink - 01-02-2017

Cool, I've been wanting to do something similar.

RE: Language parsing - phyrrus9 - 01-02-2017

(01-02-2017, 01:14 AM)God Wrote: This is well beyond my purview. You're a very talented individual though.

It works it all out by building a sort of dynamic tree at runtime and executing from the bottom up. Each statement (S) or line gets its own tree.

[Image: xfZdJr6.png]

@Ender this might shed a little bit of light. I'll write up a little tutorial on this method in the next couple days (or tonight if i get really bored)

RE: Language parsing - pvnk - 01-02-2017

(01-02-2017, 01:17 AM)phyrrus9 Wrote:
(01-02-2017, 01:14 AM)God Wrote: This is well beyond my purview. You're a very talented individual though.

It works it all out by building a sort of dynamic tree at runtime and executing from the bottom up. Each statement (S) or line gets its own tree.

@Ender this might shed a little bit of light. I'll write up a little tutorial on this method in the next couple days (or tonight if i get really bored)

I cannot wait for the release. Wink

RE: Language parsing - insidious - 01-02-2017

This is interesting! The code doesn't seem remarkably overcomplicated either, which is certainly a good thing!

I'd be interested to see what else you can do to expand on this. I've never really dabbled with creating a programming langauge or even taken a programming languages course yet though I know rudimentarily how preprocessors/transpilers work.

Your tree structure is interesting aswell. It seems as if you chose to use dynamic typing, which is an interesting choice I think. If i were to expand on this I would probably opt for stricter typing, that's just cause i'm fed up with JS, though.

RE: Language parsing - phyrrus9 - 01-02-2017

(01-02-2017, 05:46 PM)insidious Wrote: This is interesting! The code doesn't seem remarkably overcomplicated either, which is certainly a good thing!

I'd be interested to see what else you can do to expand on this. I've never really dabbled with creating a programming langauge or even taken a programming languages course yet though I know rudimentarily how preprocessors/transpilers work.

Your tree structure is interesting aswell. It seems as if you chose to use dynamic typing, which is an interesting choice I think. If i were to expand on this I would probably opt for stricter typing, that's just cause i'm fed up with JS, though.

I didn't choose to use dynamic typing, I simply made all variables of integer type. Going beyond that is a little more complicated

RE: Language parsing - bitm0de - 01-03-2017

That code could be heavily cleaned up. You could group similar constructs that use the same grammar for expressions and create proper parsing tables to ease the process of using lexical analysis + parsing. Reduce the number of hardcoded numbers you're using, and make more appropriate macros, instead of:

Code:
#define ERR() free(tmp); error = true; return 0;

If you read up on why people use do{}while(0), you'll see many discussions on this. I don't know why a macro like ERR() should free stuff on the heap, and return 0 though, that's not really self-documenting, and the way it's written you restrict it from being placed in a lot of locations in code.

I read through some of the other threads too but I didn't see any mention of LALR or LR(1) or any of the other common parsers. If anyone is really serious about programming they should look into how those work. Not only do they help you reduce boilerplate code if you implement them properly, but they're also far more scalable than the manual parsing you're doing on a 1-to-1 per syntax basis.

As a sidenote, 512 being a multiple or power of 2 is not the best number for CPU cache performance. There's a good PDF somewhere written by someone that explains a few things about this number, but I'm sure you can find it with a simple google search. I would've just chosen a PAGE size personally.

If you posted this on github I'm sure others could help you clean it up a bit too.. Real parsers aren't written like this because this will never scale. It is about tokens but you don't create any hierarchy of tokens anywhere in that code; no trees, just string manipulation and conversions.