So why not make a game engine using my language with crules as a scripting language so I can see what the API _SHOULD_ look like for the client. So i choose doom since i am terrible with graphics, i can use the doom.wad and concentrate on programming; So in the past i have written two separate 2D game engines one was a Java 2D Monkey Island clone i done for university and it was terrible since Java is a terrible platform in my opinion, then my 2nd was a port  of it over to C++/SDL and the code was terrible so don’t bother asking me about it.

Anyway’s so the long and short i am writing doom engine from scratch, my original idea was to work with an existing game engine like Crystal Space or Doomsday Engine though, Crystal Space i was put off since it already uses Python and i don’t really feel like trawling though lots of code and forking it essentially would be really more work than its worth. Doomsday Engine is the cream of the crop of Doom Ports at the moment, all others are practically the same in essence using the same ancient code and hacks from throughout the 90′s. And really the technology has changed somewhat since then. Though Doomsday Engine is pretty amazing and works well, it seems to need a massive cleanup and i don’t mean to be nasty on the developers or anything they have done a fantastic job!  And they have really made the code much cleaner than any other port about! Plus the lead developer has been really nice to me . Anyway’s Games have always been a great passion to me and really doom can’t be that hard to re-implement from scratch and really it isn’t i already have a basic game engine working using OpenGL and SDL{ _ttf, _mixer }. With audio, and fps counter and input bla bla. But i am just concentrating on differentiating the wad sprite formats etc while revising for exams and working on crules and many other bits and pieces. So in the end  i’ll just stick in an extract of the log of my game engine when it starts parsing the doom.wad and explain a little on how to parse the wad!

debug: main.c:28 -> Trying to start crldoom wad: data/doom.wad!
debug: doom.c:147 -> parsing wad file <data/doom.wad>!
debug: doom.c:80 -> is a wad file :: <IWAD>!
log: doom.c:155 -> wad file data/doom.wad has length 12408292!
debug: doom.c:27 -> debug seeking…
debug: doom.c:30 -> reading….
debug: doom.c:34 -> parsing…
log: doom.c:49 -> header:: ident -> 1145132873, n_lumps -> 2306, infotableofs -> 12371396!
log: doom.c:62 -> directory:: filepos -> 12, size -> 10752, name -> 1497451600, ident -> PLAYPAL!
log: doom.c:62 -> directory:: filepos -> 10764, size -> 8704, name -> 1330401091, ident -> COLORMAP!
log: doom.c:62 -> directory:: filepos -> 19468, size -> 4000, name -> 1329876549, ident -> ENDOOM!
log: doom.c:62 -> directory:: filepos -> 23468, size -> 6854, name -> 1330464068, ident -> DEMO1!
log: doom.c:62 -> directory:: filepos -> 30324, size -> 9402, name -> 1330464068, ident -> DEMO2!
log: doom.c:62 -> directory:: filepos -> 39728, size -> 15466, name -> 1330464068, ident -> DEMO3!
log: doom.c:62 -> directory:: filepos -> 55196, size -> 3286, name -> 1330464068, ident -> DEMO4!
log: doom.c:62 -> directory:: filepos -> 58484, size -> 0, name -> 827142469, ident -> E1M1!
log: doom.c:62 -> directory:: filepos -> 58484, size -> 1430, name -> 1313425492, ident -> THINGS!
log: doom.c:62 -> directory:: filepos -> 59916, size -> 6804, name -> 1162758476, ident -> LINEDEFS!
log: doom.c:62 -> directory:: filepos -> 66720, size -> 19980, name -> 1162103123, ident -> SIDEDEFS!
log: doom.c:62 -> directory:: filepos -> 86700, size -> 1880, name -> 1414677846, ident -> VERTEXES!
log: doom.c:62 -> directory:: filepos -> 88580, size -> 8964, name -> 1397179731, ident -> SEGS!
log: doom.c:62 -> directory:: filepos -> 97544, size -> 956, name -> 1128616787, ident -> SSECTORS!
log: doom.c:62 -> directory:: filepos -> 98500, size -> 6664, name -> 1162104654, ident -> NODES!
log: doom.c:62 -> directory:: filepos -> 105164, size -> 2288, name -> 1413694803, ident -> SECTORS!
log: doom.c:62 -> directory:: filepos -> 107452, size -> 968, name -> 1162495314, ident -> REJECT!
log: doom.c:62 -> directory:: filepos -> 108420, size -> 6948, name -> 1129270338, ident -> BLOCKMAP!

So this is just some of the output of the code i had written it parses out whats called the ‘directories’ in the file and then i also have functions to parse out whats called the ‘lumps’ from these directories the lumps are the actual data. I’ll explain more detail when i talk about how to parse this. And how to convert the music lumps to proper midi so you can play it in SDL_mixer or even in Totem if in in gnome.

So what do we need, if you have yourself a Doom.wad file lying about i am sure if you go through your old boxes you’ll find yourself a copy of doom, or if you cant find it you can find it on torrent sites but you didn’t read that here . Thing is I have bought the game like 3/4 times I am sure hehe. Lets get started.

In the Doom wad there is whats called the ‘HEADER’ which is of length 12 bytes which contains 3 * 4-byte integers which contain the data we care about, so lets do that lets make some code (i just wrote this code in like 10 min to illustrate the idea its ok does the job!):

So lets compile and test this code <remember to change the path to your doom wad>:

gcc doom_wad_test.c

./a.out

So right what does any of this code mean i just chucked it up there, i wrote it up very quickly from scratch. So as i stated earlier there is a wad header which is a 12 bytes in length which i refer to as ‘WAD_HEADER_LENGTH’ which contains 3 * 4-byte integers. So from the code we have fopen’d the wad_file and i use fread to read in the 12 bytes in a 12 byte ‘unsigned char[]‘ buffer. So to read out the integers in a neutral endian way i found a nice parse_int function in some of the wad documentation i found and in old doom source code which was GPL so it was like…. yoink… but its pretty easy to understand if you know your representations well enough but i wont go into detail.

The first integer is the IWAD or PWAD string, IWAD is what you will find mostly its the deployment version of the wad, the PWAD is a patch wad and very few of them are still around and i think it has similar structure. Anyways, you parse out the first integer like this

Remember to add null string ‘\0′, so we don’t get buffer overflow! This string is always a length of 4 so no need to mess about! And then the next integer is the number of lumps in the wad the lumps are the actual binary data like music and graphics. Third int is the ‘directory offset’ which contains pointers to where the lumps are their idents and the size of the lumps which have lengths of 16 ‘WAD_DIRECTORY_LENGTH’ until the end of the wad file! so note i find the length of the file and do:

So how do we parse out these directories? We move to the directory offset and then read the 16 bytes into a buffer, which is 4 integers this time. But still contains only 3 things means the ident strings can be longer or can be with some padding between each directory lump.

Now we know the file offset of the lump the size of the lump and the ident of the lump and we make sure to add the null terminator again to avoid buffer overflow. And then we can parse out the lumps simply just fseek and fread Done its so simple and seems like there would be more to it, yet when you think of it is very much common sense there really isn’t that many ways to pack data into a file like this! If your interested i’ll be talking about it soon… on how to convert the D_E1M* lumps which are music lumps to compatible midi tracks. And finally on how BSP works for the levels and maybe how to draw them in C/SDL/OpenGL… if you ask i prefer C over C++, yeah when it comes to building this game engine there have been times i though woo having an object would be really nice here but then i realise well no because that’s what the scripting engine is going to do… Mainly due to the fact i dont like C++ syntax it feels messy to me and C is just nice and simple

Finally I just want to wrap up with saying John Carmack has to be one of my biggest Hero’s in Computer Science along with Migel De Icaza, Linus Torvalds, Various GCC hackers… like Ian Lance Taylor But John Carmack in particular since he made Doom! But also he attended the University of Missouri–Kansas City for two semesters before withdrawing to work as a freelance programmer. Which i think that is brilliant! Since University does not teach students to be serious programmers or technology well very few do anyway and i think its great he had the balls to do that and so did Migel De Icaza and they have both done some of the most amazing things in computer science!

Anyways if your interested in Crules or CrlDoom see: http://crules.org code for crldoom is over @ http://code.redbrain.co.uk/cgit.cgi/crldoom

This is a project i love Classic Doom 3 it was Doom re-implemented ontop of the Doom3 Engine what more do you want!!!

So lets just take an extract since i am lazy: So in compilers or interpreters how do we represent:

Lets see it in a diagram first then in IR code:

Expression

So what is this? Well early on when i was getting into compiler construction when doing semantic analysis you will need to find a way of representing these semantics for your language, and you will find generally that there are 3 things you need to know: DESTINATION OPERAND A and OPERAND B. This is what’s called 3-address code. This is what sets compilers apart and can make or break them! Its what gives implementations flexibility and the ability to ‘control the flow‘ of execution! Any good compiler book will give you more detailed discussion on this topic its my main interest in compilers since this is what gives an implementation its logic. So lets give it a syntax we can write down to illustrate how mine works: it just so happens a LISP ‘ish syntax can show this very well!

Since the parser is precedence aware it will construct the tree as shown in a symbol language which looks like this if it was output ( outputting the IR like this is a future TODO ); It may look a little like GIMPLE but this IR I have as I will demonstrate more examples can encompass much much more.. ( this is a 3-address code by the way )

busy!

Though one big note its is ok for me to leave the representation like this since in my interpreter i just evaluate this at run time but how does this affect things like code-generators like proper Compilers? Well its quite simple in a sane compiler like GCC ( although most compilers will follow this idiom i am sure) introduce another layer for IR called RTL ( Register Transfer Language ). The wikipedia article lacks and most compiler books overlook this section very much since, most books are written by the academics and generally stay away from the back-end of compilers since that is the actual hard part since a code-gen is very specific to the instruction set your targeting and register allocation algorithms are very difficult and is a huge research project.  Its said in computer science the most difficult things to build software wise is a good JIT or any Code-Generator for that matter.  Anyways back to the problem lets take a smaller expression for sleep’s sake the expression:

Will generate something like:

Anyways you’ll notice in my the IR keywords such as SYMBOL_ACCESS or SYMBOL_ITEM or OP_ADD etc, these are what’s called OP_Codes and are generally represented by some symbolic hex values since its cheap to check against for the implementation to figure out what to do with symbol ‘x’; So ok thats a quick and dirty intro to Intermediate Representation’s!

I think this lolcat will sum up your feelings at the end of reading this blog post:

And i started getting out my old grunge cd’s which include my Silverchair collection and this is on of my favourite bands of all time and this is a pretty good song!

First off go and download the 750mb tarball of the Intel C compilers over at: http://software.intel.com/en-us/intel-compilers/

% tar zxvf l_cproc_p_11.1.059_ia32.tgz

% cd l_cproc_p_11.1.059_ia32

% ./install.sh

# follow all the instructions it complained i didn’t have java installed but you don’t need it. Once your finished it should all be installed to /opt/intel/Compiler

% PATH=$PATH:/opt/intel/Compiler/11.0/069/bin/ia32

% LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/Compiler/11.1/059/lib/ia32/:/usr/lib/

You should be able to run ‘icc’ but you will probably get the error: ” /opt/intel/Compiler/11.1/059/bin/ia32/mcpcom: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory

compilation aborted for test.c (code 127)”

So you need to install an older libstdc++ on debian it is:

% aptitude install libstdc++5

That should be it lets give it a test:

% icc test.c

Should work now! If your project uses a Configure.ac lets make a quick change:

Change your AC_PROG_CC to:
AC_PROG_CC([icc gcc cc cl])

This is the order in which the configure should search for each compiler. Since you need GCC installed to use Intel C++ you need to put ICC at the top of the list! Hope this helps someone!

Oh and i added the link for my hello-world gcc-frontend

$ git clone git://github.com/redbrain/gcc-sbsh.git

$ ./configure

$ make (..sleep for 2 hours…)

//what your interested in is gcc/sbsh/ * all the files in there is all the stuff you have to do to make a hello world front-end you need all those definitions etc.. i think i have broke my Make-lang.in but i’ll fix it asap. If your on a mac i posted the fix for mpfr on the gcc-help mailing list.

To compile on a mac:

$ wget -c ftp://ftp.gmplib.org/pub/gmp-4.3.1/gmp-4.3.1.tar.gz

$ tar xvf gmp-4.3.1.tar.gz

$ ./configure

$ make

$ sudo make install

$ sudo port install mpfr

$ cd gcc-sbsh

$ ./configure –prefix=<maby put a prefix for install here instead of sys-wide install> –with-mpfr-lib=/opt/local/lib –with-mpfr-include=/opt/local/include

Now it should configure properly!   http://www.nabble.com/Configuration-problem:-not-recognizing-MPFR-td23556792.html. You may find my on gcc-help pretty often now and automake bison and autoconf. I use these apps alot now and the mailing lists are really intereasting. Linux-kernel list sucks its far far too fast to follow unless you a full time-developer drizzle i dont have enough time for either KVM i think they are all over the place

And here is the git-repos of my work for the hello-world interpreter its still a work in progress but i think its ok for one evenings work

$ git clone git://github.com/redbrain/simple-programming-language.git

$ apt-get install build-essential autoconf automake libtool

$ sh autorun.sh

$ ./configure

$ make

$ ./src/simple –help

$ ./src/simple tests/helloworld.sim

Oh and i have done some massive updates to the Belfast LUG wiki in general: http://www.belfastlinux.org/wiki/Main_Page

http://www.belfastlinux.org/wiki/2009-04-06_mobile-monday

http://www.belfastlinux.org/wiki/Meetings

I will aim to keep the Meetings page updated with my email after the meeting with what happened at the Meet like in: http://www.belfastlinux.org/wiki/2009-03-25_pubbage. I will maby start a page in the future about going to LugRadio live 2009 but thats a good few months off yet..

So yeah This last week and a few days i have spent my evenings doing some writing on compilers etc… for the tutorial and playing with haskell.. I am trying to learn the curl api to make an identi.ca client written in haskell and it cant be hard because the code to do it in C is well tiny look i knocked this up in a few mins:

You can compile with:

$ gcc -o identica main.c `curl-config –libs –cflags`

So as you can see posting to identi.ca or twitter is a simple task. But haskell… well it isnt simple I do love haskell in some ways its very much the way you think of things if your doing a very long definition in Pure Math or a proof in Applied Math. Yeah i do a Math degree… i hate it a i love it just i prefer doing programming mostly.

Anyways the problem i have with the language is its becoming EXTREMLY messy in my opinion, instead of people concentrating on making haskell useful to actually make apps and wrappers to useful libraries they seem to keep adding features to the language. Like the way things work in it take a long time to get your head around but to manipulate data is why i LOVE it. So its like look at this tutorial it just explains why i love haskell http://www.haskell.org/~pairwise/intro/section1.html#part4

lolcat

Thats just an awesome idea lazy evaluation, i am getting to grips with it but i think it would be better if i get a book on it rather than reading a sticking bits of what i see around the interenet together.Its just its all very well being able to make a list and do silly evaluations on them etc.. But you need to have some good networking apis or some decent file-read-write apis there are some but its just very centric around read or getch type functions. Then GHC dam its a huge and the main implemenation of haskell, but its sooo bloated. I was toying with my own idea to make a gcc-haskell front-end but i don’t know if i like the language enough to do it, i would probably prefer to make my own version of it. But then it would take a long time to persude people to move over. etc…

Anyways its nearly 1AM and i have work in the morning Remember to post comments guys!