Andromeda - Galaxy Extension

[gexxo]

I haven't introduced Andromeda here yet, but since I see people being annoyed by GUI I'll catch up on it now.

So, what is Andromeda? Short answer: An object oriented extension language to galaxy.

Why should you use it? If you used vJASS or something similar for WC3 you will like it, because it is compareable to it, but even now in its beta stadium more sophisticated (does typechecking, finds dead code and unused stuff, does optimizations,...).

We are some map makers from germany and already did much stuff for Warcraft 3. I made Castle Fight, YouTD and GMSI, a script language to alter Warcraft 3 maps procedurally ("makefiles for maps") and I specialize in compiler construction at my studies. My mate tolleder has experience in mpq handling and blizzard's file formats. So we have some pretty good starting point for creating a hopefully good extension language.

Let me shortly introduce our main goals to you:

• Being totally compatible to galaxy, allowing for the use of 'legacy' galaxy code (all galaxy code also compiles with the Andromeda compiler, but might get optimized)
• Providing object oriented paradigmns (Classes, Interfaces, "Enrichments", Inheritance, Generics and even reflection (partly, as much as possible with galaxy code), ...)
• Deep syntax analysis for finding possible errors (dead code, unused variables / functions, ...)
• Comprehensive Code optimization (function inlining, removing comments and whitespace, resolving constant expressions, ...)
• Convenient constructs that are found in most languages but missing in galaxy (++/--, for loop, for each loop, casts, block comments, overloading ...)
• Help for debugging, by allowing to override native (and thus track) function calls and other debugging stuff (we are even working on a debugger for it).
• Preprocessing for adding some code in debug mode for example (#ifdef, #define, ...)
• Speed. Andromeda was coded to be fast. Even if it is written in Java, it can parse, check and compile the native libraries (around 3000 lines of code) in only 0.3 seconds on my machine.
• We sticked to the JAVA specification wherever possible. So the language syntax (especially the object oriented one) is copied from java, so if you can write a java class, it is probably also correct Andromeda code. However, we added some non-java stuff like accessor methods and some syntax to stay compatible to galaxy (structs, typedef,...).

The language is totally written in JAVA so it is (hopefully) platform independent. Even the mpq handling code (not yet included, but already working, more or less) is completely written in java without any storm.dll or sfmpq.dll that would break the platform independance.

We are also cooperating to create an IDE for it, which has all the cool stuff you want (code completion, parsing at realtime (i.e. see an error after you did it, not only when you press the compile button, as known from eclipse).

Roadmap that shows an overview of all features and how far they are implemented. Also download is available here (scroll down to the bottom).

Specification: Detailed description of all features, their syntax and their semantics. Every feature that is described here is already working in the current release.

Any comments welcome, I'll try to answer all question that you might have .

 

[Element of Water]

Why "Andromeda"? Just curious. I'd have called it "Galaxy++", but I'm not very imaginative

Anyway, feedback...

Deep syntax analysis for finding possible errors (dead code, unused variables / functions, ...)

That actually sounds really annoying. Sometimes you have function and variables there that you're in the process of making but haven't got round to using yet, and you want to check that everything is right so far. Unused things definitely shouldn't be errors, but maybe you could add a "warning" and delete them during optimization.

Comprehensive Code optimization (function inlining, removing comments and whitespace, resolving constant expressions, ...)

Don't do that by default, it would just increase saving time. Make it an option, maybe "Release Mode".

We are also cooperating to create an IDE for it, which has all the cool stuff you want (code completion, parsing at realtime (i.e. see an error after you did it, not only when you press the compile button, as known from eclipse).

This sounds the best feature of the lot . It would be great if you integrate it into the Galaxy Editor somehow.

Overall... wow. It sounds like a lot of work, but when it's finished it'll be the greatest thing ever to happen to SCII.

 

[gexxo]

thx for feedback, however about your first two points:

Even right now the stuff you mentioned can be changed in the config file . You can chose for every detail between those things:
REMOVE: remove the unused stuff without producing warning /error
WARNING: produces warning
ERROR: produced compile error
IGNORE: does nothing

The best thing is: You can create different run configurations.
For example, a DEBUG and a RELEASE configuration (already built in the default config).
In debug mode warnings a given and no inlining is done for example, in release mode unused stuff is removed and inlining is done as far as possible.

 

[s3rius]

I've been following the Andromeda project for a while now and I like what I read

One question though: how's the progress of the GUI? I'm uncomfortable with command line input..

 

[gexxo]

well GUI is not so far. This is because I don't know how the final usage will be. If you use a special IDE, it will just call command line Andromeda, so no GUI needed there. However, for standalone and for the editor, I need some GUI.

But before I create the GUI I want andromeda to be able to read from / write into MPQ files, otherwise it is no use anyway (since you would need an MPQ editor to inject the final MapScript.galaxy into your map). We re working at that at the moment.

But got nice news:
I just used the new set of native galaxy files for compiling, which is around 15000 lines of galaxy code in 69 (!) files. It was parsed and semantics checked in only about half a second . Looks like the performance will stay good even for bigger scripts (before we only tested with the old native list which was only around 3400 lines).

 

[Grey Nightmare]

I like the name... it fits... though 'Universe' would be cool, as that's the 'next' step up from a galaxy

Amdromeda is a particular galaxy I think?

 

[gexxo]

I like the name... it fits... though 'Universe' would be cool, as that's the 'next' step up from a galaxy

Amdromeda is a particular galaxy I think?

indeed, it is our neighbor galaxy (nearest). Universe sounded strange, as well as Cluster...

 

[Element of Water]

I guess "Milky Way" sounds like a chocolate bar, so you can't use that

 

[Anachron]

*currently learning Andromeda*.
Thanks, gexx.

 

[Flame_Phoenix]

Galaxy is not even complete, and it is Beta as well. What makes you think that your features will not be included on Galaxy final release?

Anyway, I like the part where you say it is platform independent, this means I can compile the language using Linux.
And how do you plan on hacking the WE ? By memory injection as well?

 

[Element of Water]

Galaxy is not even complete, and it is Beta as well. What makes you think that your features will not be included on Galaxy final release?

I'm pretty sure the Galaxy syntax won't be changed / added to. They might add new natives or make changes to existing natives, but Andromeda won't make a difference to them anyway.

And how do you plan on hacking the WE ? By memory injection as well?

Lol, still calling it the WE. It's the GE now
I don't think he mentioned injecting it into the editor. I guess it will just compile and import an external map script?

 

[Flame_Phoenix]

I'm pretty sure the Galaxy syntax won't be changed / added to.

Don't be sure. They did it with JASS, they can do it with Galaxy.

I don't think he mentioned injecting it into the editor. I guess it will just compile and import an external map script?

MMm, so if I make a map using Andromeda, how am I supposed to run it after? And how am I supposed to test it (by running it) easily? (in JGNP you simply needed to press f8). This debugger thing, will it be like the debugger pipedream made ??

Too many questions ...

 

[elfian]

This would be single best shit EVAR!!

 

[gexxo]

blizzard already stated that galaxy will not be object oriented, so the most important parts, classes and enrichments, won't get obsolete.

And for the other syntax extension: If blizzard builds one of them, it makes compiling easier I will just remove my emulating constructs and use this one. There is no problem in adapting future features of galaxy .

However, I fear that not much new stuff will come in galaxy. Especially I fear that the 2^21 (2 mb) byte code & variable space limitation will stay forever which would be the worst thing by far (we debugged the virtual machine that is running galaxy while the game runs, and the 2^21 limit seems to be inherent to the virtual machine operation structure, so they have to change their VM drastically to allow more than 2^21 bytes. I will soon publish a request to blizzard to do this).

 

[Flame_Phoenix]

And for the other syntax extension: If blizzard builds one of them, it makes compiling easier I will just remove my emulating constructs and use this one. There is no problem in adapting future features of galaxy .

As long as you never break backwards compatibility I guess it is OK.

Please answer my other question btw.

 

[Na_Dann_Ma_GoGo]

Nice.
After the first lines I was like "Euh, well I doubt this will work well already...".

Then I checked your user name and my thoughts immediately changed ;P

GMSI and your Wc3 Maps are just awesome. Good luck with Andromeda (interesting name choice), hope it turns out well.

 

[gexxo]

As long as you never break backwards compatibility I guess it is OK.

Please answer my other question btw.

It won't break, since it just compiles to galaxy. Even if I change the way things get compiled, stuff compiled before will still work (just with other galaxy constructs).

@other question:
You meant the editor hacking? Well I don't know exactly, but since JNGP did it for world editor and starcraft editor is not very different, I think this should be no too big problem.

 

[Flame_Phoenix]

So, let me see if I get this right: your overall plan is to inject a java IDE via memory injection into Galaxy Editor. For some reason this seems too heavy to be efficient... I guess...

 

[gexxo]

So, let me see if I get this right: your overall plan is to inject a java IDE via memory injection into Galaxy Editor. For some reason this seems too heavy to be efficient... I guess...

nono

JNGP also doesn't inject jasshelper machine code into the editor process. It just inserts a small hook which calls the program via execute process.

So what you actually do is injecting a hook that is executed whenever you press the "test map" button. First, the editor saves the file, then the hook jumps in, holds execution and runs Andromeda over your mapfile. Then SC2 opens with the map with got the altered MapScript.galaxy from Andromeda.

 

[Anachron]

Nice, Gexxo, how are the inlined functions going?

 

[Flame_Phoenix]

Your project seems quite interesting from the technical point of view. I wish I could learn more and somehow help you. Does Galaxy support 2D arrays? You could add that to Andromeda, like Vex added to vJass.

 

[Anachron]

Yes, Galaxy supports up to 4D Arrays.

 

[Dr Super Good]

I hate to break it to you guys, but the editor is only a beta. As such it is subject to be changed anywhere from slightly to toatally.

Blizzard themselves admit the editor currently is incomplete, as such you might find many of the features you are after (real time syntax checking) may eventually get added.
Until the game is actually released, I would not attempt to hack the editor.

 

[gexxo]

Nice, Gexxo, how are the inlined functions going?

Not advancing at the moment, because this feature is not necessary at the moment. I first want to make everything work that adds new functionality. Making function inlining work can also be done later...

I hate to break it to you guys, but the editor is only a beta. As such it is subject to be changed anywhere from slightly to toatally.

Blizzard themselves admit the editor currently is incomplete, as such you might find many of the features you are after (real time syntax checking) may eventually get added.
Until the game is actually released, I would not attempt to hack the editor.

Realtime syntax checking won't be added, I am 99% sure of that .

 

[Anachron]

I see. What about destructors?

 

[gexxo]

they will come soon, since they add functionality and polymorphism is working

 

[Anachron]

Great, alright.
I need help about implementing Andromenda, how can I make it run automatically on save? Is there any way?

 

[gexxo]

Great, alright.
I need help about implementing Andromenda, how can I make it run automatically on save? Is there any way?

not yet, we are the hell working on the mpq module which you need to inject the code into the map. Since we could only test since the editor is out, we will still need some time (but we re quite far).

I must confess that working productively is not yet possible with the current release but we are trying to get a working version asap.

 

[Dr Super Good]

You need to do a memory scan and inject your code into the editor like JNGP did. This requires you going through the editor with a disassembler fidning the right places to inject. This is why JNGP has problems with patches. I am prety sure I saw some kind of heck syntax button in SC2, but then again I am still very new to it.

 

[Element of Water]

I am prety sure I saw some kind of heck syntax button in SC2, but then again I am still very new to it.

There is a "Compile All" button, but it doesn't work for custom scripts in GUI, just in that script editor that doesn't let you use the scripts you write

EDIT: I only just realised we can actually download it now, but I can't work out how to use it. I tried running the .jar file and nothing happened, I tried calling it with a batch file followed by the name of a galaxy file, nothing happened... how do we actually use it?

EDIT2: It turns out I was using it right, but it didn't do anything since I had an error. You should at least make it say "Error on line X -- stopped compiling" even if you can't output proper error messages yet.

Also, the outputted scripts using classes give syntax errors in the galaxy editor, because pointers are no longer supported as of patch 9. I guess you'll have to resort to using the array index as a pointer like vJass does.

 

[Zoxc]

My compiler parses and validates the native library in 36 ms, some of which is probably I/O overhead.

It also integrates nicely into a IDE, as seen here: http://img694.imageshack.us/img694/7049/mwe6.png

</hijack>

With regards to backward compatibility Blizzard did remove pointers, did you update Andromeda already?

Also I do find a few thing odd with the language:
- private and public per-function which usually results in prefixes on most functions.
- the strcall and inline keyword, I feel like these would be better as a form of attribute, they're just hints for the compiler anyway.
- the mixing of include and import, I see no reason to do this, nor how import would be able to find the right file to import in the first place.
- I don't know if curly brackets are required for accessors, but they shouldn't be.
- you seem to have added a lot of keywords for things that probably will never be used (exceptions much?)
- it seems like the comma separator can only be used in variable declarations and for loops, which is a tad inconsistent.

Personally I'm probably not going to write any extensions until Galaxy stabilizes, the removal of pointers pushes the potential extensions in the direction of Java/C# though.

 

[gexxo]

My compiler parses and validates the native library in 36 ms, some of which is probably I/O overhead.

It also integrates nicely into a IDE, as seen here: http://img694.imageshack.us/img694/7049/mwe6.png

</hijack>

With regards to backward compatibility Blizzard did remove pointers, did you update Andromeda already?

Also I do find a few thing odd with the language:
- private and public per-function which usually results in prefixes on most functions.
- the strcall and inline keyword, I feel like these would be better as a form of attribute, they're just hints for the compiler anyway.
- the mixing of include and import, I see no reason to do this, nor how import would be able to find the right file to import in the first place.
- I don't know if curly brackets are required for accessors, but they shouldn't be.
- you seem to have added a lot of keywords for things that probably will never be used (exceptions much?)
- it seems like the comma separator can only be used in variable declarations and for loops, which is a tad inconsistent.

Personally I'm probably not going to write any extensions until Galaxy stabilizes, the removal of pointers pushes the potential extensions in the direction of Java/C# though.

well I didn't know that they removed pointers. This makes things really complicated... (how to hand a class/struct to/from a function for example). What about a petition to Blizzard asking for pointers again and all other problems galaxy has at the moment => here do we go

 

[Zoxc]

My "petition" is possibly the reason they removed pointers in the first place. See: http://forums.battle.net/thread.html?topicId=23425634566

 

[gexxo]

I like yours, (but I think you got too far with Closures, blizzard is even struggling with function pointers, do not make them dizzy )

check out my PM!

 

[gexxo]

v0.0.11 released

This is a huge milestone in the development of Andromeda because v0.0.11 is the first version that can actually be used productively. Not only is it now able to read Andromeda code from map files and write back the compiled galaxy code into them, but it also comes with a small, yet very simple GUI to run Andromeda onto your maps. The GUI however is not finished yet, so expect updates for it soon.

Also classes work again now (without using pointers anymore ;( ) and static initializers and destructors were added, too.

An installation and usage guide can be found here.
The download itself can from now on be found in our resource database.

Since the MPQ handling is still in a beta status, please remember:
Make a save copy of your map before running Andromeda on it, especially when saving to the same map file!


Here is the complete changelog:
-Classes no longer compile to pointers but instead use an array indexing system for dynamic allocation
-Fixed many small errors
-Fixed mixed up line breaks
-Added static initializers (check specification)
-Added destructors (check specification)
-Andromeda is now able to read code from maps and write it to maps / combine it with the script that is present in a map
-Added a simple GUI to execute scripts. This GUI starts up if Andromeda isn't started with parameters
-Added new language class System (a/lang/System.a)

 

[tagg1080]

I would seriously like to know what is wrong with the language we have been given?

I feel like you want to write this, and havn't actually had experience using the given language to really know what needs/doesn't need to be changed.

I am just curious.

 

[Flame_Phoenix]

I would seriously like to know what is wrong with the language we have been given?

Galaxy is an imperative language Value based -> this means that it doesn't follow the Object Programming paradigm which is superior than the Value programming paradigm.

Gexxo is trying to add the object oriented paradigm concept to Galaxy so it evolves into a more modular, easy and portable stage (thus becoming a better language).

Galaxy is nothing but an improved version of JASS. Gexxo has a lot of experience with JASS and it's globally known extension - vJass - so yes, he knows what he is doing.

You are the one who should experiment or at least read things before giving an opinion.

 

[tagg1080]

Galaxy is an imperative language Value based -> this means that it doesn't follow the Object Programming paradigm which is superior than the Value programming paradigm.

Gexxo is trying to add the object oriented paradigm concept to Galaxy so it evolves into a more modular, easy and portable stage (thus becoming a better language).

Galaxy is nothing but an improved version of JASS. Gexxo has a lot of experience with JASS and it's globally known extension - vJass - so yes, he knows what he is doing.

You are the one who should experiment or at least read things before giving an opinion.

OO is not always better...

I just don't see the need to work on a project of this magnitude before the full editor is even out yet...

 

[gexxo]

OO is not always better...

I just don't see the need to work on a project of this magnitude before the full editor is even out yet...

if you don't like oop and want to write useless conversion stuff like StringToText over and over again, then yes, you might be better off with galaxy. Anyone else is better off with andromeda. Many people used vJASS with wc3 to overcome the limitations of JASS. And as Flame Phoenix already stated, galaxy is just JASS with C-syntax and almost unusable structs. They didn't really add much functionality.

The need to work on it right now: to ensure that it is more or less stable when SC2 is released so everyone can use it from the beginning on.

 

[Element of Water]

Galaxy is an imperative language Value based -> this means that it doesn't follow the Object Programming paradigm which is superior than the Value programming paradigm.

I've never heard it called "Value programming" before. Did you mean "procedural"? Anyway, OO is not always better, as tagg1080 said: procedural programming is quicker and simpler to program with, and the main advantage of OO is organisation, so procedural programming is probably better for small projects and OO for large ones.

Andromeda does add other improvements, however, so I'd always use this over regular Galaxy.

 

[Flame_Phoenix]

OO is not always better...

Lol, get real kid.

I've never heard it called "Value programming" before. Did you mean "procedural"?

The portuguese translation is "valued programming" (meaning that you program by continuously changing the state of variables and not of objects). I am not sure how to spell the correct term in English because I never heard it.

The main reason why any OOP language is superior to any procedural language (as you say) is because OOP languages support the procedural paradigm themselves. This way they are as fast as, and more modular than simple procedural languages. See the example of C and C++, they are equally fast and C++ is way more structured and advanced. In fact, C++ also supports any syntax C supports. This is a poor example. You can see Java (per example) which supports the OOP and the procedural paradigms. Other examples may be the smaltalk language. Note that this rule only applies to imperative languages and not to functional languages like Ocaml and Prolog ( I don't fully understand their programming paradigms so I wont comment them).

 

[PurplePoot]

I think several people in this thread have problems with their brains being missing.

First, OOP is not the strictly best paradigm, nor even the best imperative paradigm (I like that you distinguished between imperative and functional languages, because someone who says one is strictly better than the other is provably wrong). Even if you end up with a language which is a strict subset of another (C vs C++, for example), the superset is not necessarily better because bad additions which become adopted as standards can hurt development, readability, and maintainability.

Second, OOP has nothing to do with typecasts.

Third, (int)"42"; is a horrible horrible construct which should have never been created because it makes absolutely no sense and means a very different thing that what you are making it do.

 

[Flame_Phoenix]

I think several people in this thread have problems with their brains being missing.

Great, because flaming people is always the best way to solve problems.

First, OOP is not the strictly best paradigm, nor even the best imperative paradigm

What is the best paradigm then?

I never mentioned typecasts nor (int)"42".

I don't get the main point... are you defending OOP and Andromeda or not? xD
The only con of an OOP paradigm is that sometimes it can make the program slower (not slow enough to be of importance). However, if you want to be an efficiency freak then go ahead and start learning NASM, it is even faster then C or any other language, you can't get any faster.
I believe that Andromeda is a compromise between Galaxy's natural efficiency / functionality and the OOP paradigm, putting together the best of the two worlds, which is why I support Andromeda.

Btw, nice to see you again Pooty

 

[elfian]

I think several people in this thread have problems with their brains being missing.

First, OOP is not the strictly best paradigm, nor even the best imperative paradigm (I like that you distinguished between imperative and functional languages, because someone who says one is strictly better than the other is provably wrong). Even if you end up with a language which is a strict subset of another (C vs C++, for example), the superset is not necessarily better because bad additions which become adopted as standards can hurt development, readability, and maintainability.

Second, OOP has nothing to do with typecasts.

Third, (int)"42"; is a horrible horrible construct which should have never been created because it makes absolutely no sense and means a very different thing that what you are making it do.

Why is int("42") a bad construct?! WTF?!

 

[PurplePoot]

What is the best paradigm then?

You're making the assumption that there is one.

I never mentioned typecasts nor (int)"42".

I was referring to gex there.

I don't get the main point... are you defending OOP and Andromeda or not? xD

I hate a lot of what has been done with Andromeda, and I like OOP as a paradigm but not the one paradigm to rule them all.

Also, I know ASM, and I have no desire to program in it more than is necessary.

Why is int("42") a bad construct?! WTF?!

You clearly have no idea what a typecast is if you think that (int)"42" (not int("42")!) is not a bad construct.

 

[elfian]

Yes I do know what a typecast is.

You have to explain two things now:
1. Why is (int)"42" bad?
2. What is the difference between (int)"42" and int("42")?

 

[Flame_Phoenix]

You're making the assumption that there is one.

Lol, how nice. I was hoping you would say something like the Automata paradigm or something else, thus enlightening me with something new . I think that what you mean is that all paradigms are good depending on the use you want to give to them and depending on the problem you are trying to solve.

Also, I know ASM, and I have no desire to program in it more than is necessary.

I have no desire to program in it at all xD
Good choice my old friend.

Off-topic: Please forgive me for my ignorance in this matter. Recently I heard about an interpreted language that used used all over the world and that is used to code games as well. This language is LUA.
Now, I know that having LUA is bad (because it is interpreted and the engine can't run other interpreted languages but Galaxy), so it popped to my mind: why can't we make some sort of compiler that converts LUA code into Galaxy code ? Having in mind LUA is widely used perhaps it is not a bad idea. What do you guys think?
Sorry if the idea is dumb xD

 

[elfian]

You cannot convert Lua to Galaxy. There are a lot of reasons like:
1. Lua does not have variable types, it's all just one variable type like in JavaScript.
2. In Lua every variable can have member variables and functions.
3. Functions can be stored inside variables.
4. ... so on

They are completely different. You may only attempt to preparse somehow Lua to Galaxy, but that would be a great amount of work and really won't be worth it.

 

[tagg1080]

Yes I do know what a typecast is.

You have to explain two things now:
1. Why is (int)"42" bad?
2. What is the difference between (int)"42" and int("42")?

Do you understand the amount of crap that has to be done to make "42" into an integer?

To the untrained person, it seems simple to just make "42" into 42.

This is very much so not a "simple" process.


And what in the hell is: int("42") ??
They are using Java to write this thing, eh? So what happened to (int)"42", which would be the closest thing to Java syntax, but most definitely would not compile.

The implicit casts are what they are for a reason, and the explicit casts are what they are for a reason. I don't see the need to mess that up.

 

[Element of Water]

Ehh, (int)"42" works in Andromeda, if that is what you're trying to get at.