So hey, its been a little longer than i intended to update my blog! But i’ve been busy, so i have got rid of my personal work being on repositories all over the show. So I have made:

There will be public git access soon, but you can download the [...]]]>

So hey, its been a little longer than i intended to update my blog! But i’ve been busy, so i have got rid of my personal work being on repositories all over the show. So I have made:

• http://code.redbrain.co.uk

There will be public git access soon, but you can download the snapshot tarballs of the branches. And I have a wiki too there isn’t anything on there yet but it will give me room to stop putting long tutorial article’s as pages on my blog to proper wiki article’s:

• http://wiki.redbrain.co.uk

So Compilers, Interpreters and programming languages are my interests… *yawn! So let me show you my interpreter pipeline for my interpreted language ‘CRULES’:

Crules

So I have to admit that’s the most impressive art I ever done (I know I’m lame ). Its my pipeline for my programming language, but I guess a lot of programming languages would follow this kind of model/architecture. So it got me thinking this is nearly the basics for any semi-serious interpreter or ‘a’ language so lets talk though it.

I am still not going to talk about the details of my language because its not ready yet. Other than I am doing fun things like, so far the only data-structure I let the developer have is lists, so if you do something like:

% foo := { 2,3,4,5,6,67,7}

#is a list and strings are lists!

% str := “bla”

#is the same as

% str := { ‘b’, ‘l’, ‘a’ }

It works a lot around the shell so you can also do:

% foo := `ls`

#it produces a delimited list of strings with the delimited as carriage return or new line etc.

Though note I am going to implement an api for arrays. But not yet, because lists are going to cause problems unless I implemented a table/array of accessors to the elements in the list so you can do fast access to the elements because it doesn’t scale no matter what anyone says when say you are doing image processing on a 1024×1024 list… accessing the elements isn’t going to scale.

Anyway so lets get back to looking into the pipeline of it: So first a user creates a program lets look at the helloworld.crl:

rule main()

{

echo “Hello World!”

foo := 5 + 2 – 7

}

all := main;

So first my interpreter will start lexing this using LEX, into tokens like: rule, main ‘(‘ ‘)’…. and then the parser starts applying the grammar implemented using YACC. So I start producing symbols to build a symbol table the data-structure looks like this its basicly a more high-level 3 address code:

So first it creates the rule definition! For now just think as rules as entry points to a program so you can add more identifiers to that ‘all’ list not just main. It has a deeper meaning but I don’t want to talk or show this until I have the interpreter and language definitions more concrete!

So the the rule ‘main’ creates a new symbol like this:

//the symbol takes the identifer of the rule identifier

symbol->identifer = ‘main’

//its a rule symbol

symbol->symType= _SYM _RULE _

symbol union_a is the link to the first symbol in that block of statements and union_b is NULL because I don’t need it until I have rule dependencies it needs to list.

symbol->next= NULL, because it is just a definition

Then each expression like ‘ echo “Hello World” ‘, creates a symbol and its next = the next expression like the ‘ foo := 5 + 2 – 7 ‘. This creates an interesting one, because this can’t be reduced to a 3-address code until you split it up, for now I just assume the developer wants it to run as 5+2=7 then 7-7=0; But I am going to add precedence later. So you can be more expressive to have nested expressions in brackets to be executed first etc..

But yeah this expression is split up into 3 symbols for now, its an over all variable Assignment but it depends an expression of 5+2 as a temporary symbol then this temporary symbol – 7 then foo = this new temporary symbol and I can push these dead symbols to the garbage collector.

When a symbol is allocated, it is allocated memory of a preallocated memory heap of default 32MB, it can be extended or shortened if needs be, but it allows me to control and see how much memory is being occupied at anyone time in the runtime of the interpreter. Any symbols to be kept such as rule definitions, function definitions or variables are kept on one massive stack. So ok, that’s a problem if you need to search for them, the answer I found was having 3 other tables that link to the symbols in the stack. When any link is broke it becomes garbage, so when I have a new variable re-assignment I can break that link and remove the entry in the variable table it still exists in the symbol stack but the Garbage collector is always called on EXIT and the exit of any block statements like loops functions etc, so when you pop your way out of the stack you can have local variables instead of everything global and you have conflicts which will cause a lot of problems. Also the garbage collector is implemented as a pthread so you have have to pause for several seconds waiting for it to free the garbage. I haven’t done this yet because its going to be very awkward to keep the concurrency, the pthread mutex api should handle a lot for me but its still going to be complicated. I have thought I might like to try implementing a lot of different internal using OPEN/MP to make it even speedier, but that won’t be for a while.

And then I have been playing with LLVM to create a JIT execution system, you can see my posts on ‘Managing the JIT’ on comp.compilers, I have posted there quite a few times before, its a great list of basically all programming language, compiler and interpreter experts in the world!

So I have a variable table that links to the variables in the symbol stack and the same for the rules and functions. This speed up searching for specific identifiers.

So that’s the basics of parsing into symbols I guess, it pains me to say but many people thing a programming language is mainly a parser is so wrong the parser just lets you build your symbols the execution and management of these symbols is much, much more complicated. That’s not to say implementing a simple programming language is difficult, the problem I see a lot is people are scared of doing it because the vocabulary is huge! I mean if you start working at an interpreter from first principle’s I’ll guarantee if you have half a brain you’ll come to a similar architecture as this.

I found it difficult at the start when I knew anything in a programming language can always be brought to 3 things and I got really confused when people talked about symbol tables, 3 address code, IR (intermediate representation) and parse trees or syntax tree’s. I think a lot of developers have their own prefered meanings to them, but it is mainly a barrier of academics trying to make the subject more complicated than needs be, although when you go at this from first principle’s you’ll will soon learn .

So thats the main focus of my personal work but I am now also soon to be an approved GNU/gcc and GNU/automake developer! Can’t wait, got my FSF paper work signed and sent back it may arrive in Boston soon I guess but yeah I am excited. I am updating the automake documentation to show the best way to handle multiple LEX and YACC in one program. Its something only one of two people do but its a problem none the less and to make it portable is the actual problem. But I have the solutions to it now .

Got your attention

Thing is I would really love it if I had an AIX/FreeBSD/Solaris machine to do more testing on because now that I am doing work on GCC it would be so helpful to have more machines to do testing. But i’m ok for now, so far in GCC I am doing regression testing to find a regression I can tackle, its extremely difficult some are defunct on the bugzilla some are ancient and not perused. And the rest is very active but for MIPS or SPARC, and I don’t have a scratchbox setup to even test that yet. Would just be great to get my hands on even an old ibook for a power-pc cpu or an old IBM p-server. If anyone knows of any available I would be really grateful!

Ok this has become a very long post so I’ll shut up! . See you guys from Beflast LUG on Wednesday! One last thing i want to link you to 5 of my favourite bands and they are all off Jamendo:

• http://www.jamendo.com/en/album/26656
• http://www.jamendo.com/en/album/34523
• http://www.jamendo.com/en/album/42122
• http://www.jamendo.com/en/album/34714
• http://www.jamendo.com/en/album/3661

But on the plus side i have alot of things working out which is a first for me in 2 years and writing a small python [...]]]> So yeah its been a while i need to kick my arse in gear i guess but i have been so damn busy with work and no sleep

But on the plus side i have alot of things working out which is a first for me in 2 years and writing a small python build bot and regression tester for drizzle to mail daily reports to the mailing list…

Anyways i have this week off work to rest so i’ll try and blog up a little i should maby hopefully have a my openmoko this week which will be exciting

But yeah some coding time and music to relax me g33k

poop cats

So then i thought it better tell you what was going on there, the pics are of my new beer glass! and yeah the concert pics [...]]]> Hey so yeah i didnt really leave much of an explantation in the last post.. tbh i was drinking and half asleep writing up wiki pages

So then i thought it better tell you what was going on there, the pics are of my new beer glass! and yeah the concert pics are when i saw Joe Satriani a while back on of my favourite artists of all time!

Then the last pic was me doing math because i accidently added that pic to be uploaded… :S … did i mention i was drinking?! I have been mega tired and stressed this week and i don’t totaly feel myself yet but its getting better I just want some time tomyself this weekend to get some work done on drizzle but i am so tired when i get in from work i have had to do alot of work for the Local LUG here this week, i’ll you all know more about what,when and why soon as i know whats going on.

I just been learning alot of stuff to use gdb properly and autoconf and i want to put up a tutorial on it asap because its soo hard to get modern tutorials on these old..ish tools but they really are the best.. some people prefer cmake instead of autoconf but i reckon that autoconf is more portable to any OS/Kernel rather than cmake is just a dirty way to have the preprocessor definitions to have the configuration style of stuff in your code.. for only the normal OS’s like Linux and Mac and prob windows if you can go through the pain of cygwin!

One thing i would really like it you guys would test my boyer Moore algorithm: http://redbrain.co.uk/?p=157

I think its ok, but i don’t think its fully optimized as it should be. So yeah just to leave on the a quick note from lolcats… i am addicted i think!

Totem Cats!

So yeah just a really big shout out to ghostfreak! He has been a great influence in my life! And he rocks