What Does Everyone Want In A Compiler?

[Trigger.edge]

joining at the loop war >:3

while ((u = FirstOfGroup(g)) != null)
    // ...
endwhile

or while (u = FirstOfGroup(g))

more ideas more ideas more ideas =)

 

[Almia]

or you can do such :

repeat
     // blah blah
until ( boolean )

 

[Waffle]

or you can do such :

repeat
     // blah blah
until ( boolean )

we already have that..

loop

    exitwhen boolean
endloop

and i like the raw jass version better, doesnt mandate silly extra parens or other warts not usually needed in jass conditionals

 

[Nestharus]

Question..


Can anyone that knows Java help me code this? It's really hard : (, lol

 

[Waffle]

yeah, i could help.

well the other idea includes writing another backend for pypy and this sounds far less painful anyway

 

[Ruke]

Question..


Can anyone that knows Java help me code this? It's really hard : (, lol

i'm seriously considering rebuild vjass with antlr (reading the definitive antlr 4 reference) (just for fun and as a learning project, nothing serious). if you want, send me a message, maybe we can work out something cool _/m/

 

[Nestharus]

Sure, but I need to have this done, or a good portion done, by May 1st ; ).

 

[Dr Super Good]

I know Java but do not know much about parsing compilers lol.

 

[Nestharus]

Here is the architecture for the Symbol Table : |

https://drive.google.com/file/d/0B7tTsQxizxPSdmVvZTlmd3BnLVk/view?usp=sharing


My like 50th attempt, lmao. It's shaping up very nicely.

Use draw.io to open it



Here is an image

 

[Dr Super Good]

Hard to understand what much of it means/does.

I think your are meant to have a separate parser system which you then data feed in the parse tree. The data used by the parsers links it to the compilation unit at specific sites. The advantage of this approach is that both the parser and compilation unit could be recycled to make separate languages. In fact changing the syntax for the source language would be as simple as changing the data used to link the parser to the compilation unit.

 

[Nestharus]

Get on Skype so we can talk


The lexer, parser, and symbol table are all separate. What I put up was the symbol table.

 

[Ezekiel12]

I was originally going to focus on doing vJASS, but I decided to save it for later since it's such a mess and just do what I call xJASS,

Maybe I missed some opening post for this but what does all of this mean? "focus on doing vJASS"?

You're building just another thing that is pre-compiled to normal JASS, right? If so forgive my directness but why? If you want to advance wc3, there are better things you can do with your time.

 

[Nestharus]

Absolute generic symbol table design finally done

It can support any language and does not have anything in it that is absolutely specific to a language feature o-o. This means that it can support any language absolutely to spec . No extra unneeded features.

https://drive.google.com/file/d/0B7tTsQxizxPSdmVvZTlmd3BnLVk/view?usp=sharing

 

[Nestharus]

Ok, next amazing bit of news


As of now, here is the status of the Symbol Table, This is 26/32 finished and only 5 things not coded o-o. SymbolType will be the last large class >: D. Well, not very large.


green = done
yellow = testing
red = not done

01. Symbol
02. ISignature
03. SignatureString
04. SignatureNull
05. SymbolPrivilege
06. SymbolPrivilegeSet
07. SymbolGate
08. SymbolPath
09. SymbolPathSet
10. SymbolResolve
11. SymbolQuery
12. SymbolPathSetLinkedList
13. SymbolPathSetHashMap
14. SymbolVariable
15. SymbolScope
16. SymbolType
17. SignatureFunction
18. SignatureClass
19. SignatureGeneric
20. PatternMacro
21. PatternInternal
22. PatternSharedScope
23. PatternGlobalScope
24. SymbolPrivilegeLocal
25. SymbolPrivilegeRegional
26. SymbolPrivilegeSetBase
27. SymbolPrivilegeSetLocal
28. SymbolPrivilegeSetRegional
29. Public
30. Protected
31. Private
32. All

 

[Waffle]

wow, you work fast kudos!

 

[Waffle]

in case anybody cares..
https://www.youtube.com/watch?v=-dMBcqwvYA0&ab_channel=ng-conf

its a talk explaining the differences in angular 1 vs angular 2.
and it explains the reasoning as well.

I think some of the considerations may be useful.

EDIT:
this one's about optional type annotations in python.
https://www.youtube.com/watch?v=8QIYPws-51A&ab_channel=devbychannel

 

[Trigger.edge]

It would be very useful that could init structs in global declarations:

globals
    private Table ta=Table.create()
endglobals

then when compiling the srcipt would have to be like this:

function xJassInitStructsGlobals takes nothing returns nothing
    set ta=Table.create()
endfucntion

what do you say?

 

[Waffle]

It would be very useful that could init structs in global declarations:

globals
    private Table ta=Table.create()
endglobals

then when compiling the srcipt would have to be like this:

function xJassInitStructsGlobals takes nothing returns nothing
    set ta=Table.create()
endfucntion

what do you say?

i tend to agree, if you keep distinct globals block (sounds like you will) then this would be a very useful feature.

often times i have written init functions that nothing(or little..) else than init some global data structures.

Also adopting this would make it easier to reason about the order things get executed. in general you cant know for sure the order init functions are executed and sometiemes it is sueful to access data structures initialized by another function (eg different libs interacting, sharing configuration data etc.)

so if all globals get initialized before the rest of initializer code it would be really neat

 

[SA Dashie]

It'd be nice if you can initialize any function off of global decelerations

 

[Bribe]

Fun fact: you can use the "key" type for JassHelper to generate a unique integer during map initialization. I take advantage of this in the Table resource so that you can use keys instead of using the Table.create() method for "constant" Tables.

 

[Waffle]

hmm well its is a bit hacky.. so i dont really like it.
but given the amount of boiler-plate the nice way would require i approve on grounds of practicality but with heavy heart...

 

[Nestharus]

Hey guys, I made a presentation of what the symbol table is for anyone that's interested. This is only a small portion of the architecture >: ).

This has lots of nice images. No UML diagrams, just good intuitive images that show what's really going on.


Of course it is for a presentation, so there are holes in the slides ;D. However, the diagram of what the symbol table looks like is pretty awesome.


I'm talking to you Ruke >: o

 

[Nestharus]

Final road map for the symbol table

I have essentially 2.5 days left o-o.

Classes to Review: 1 					(to finalization)
Classes to Test: 4						(to final review)
Classes to Fix: 3						(to test)
Classes to Update: 8					(to fix/test)
Classes to Create: 1					(to fix/test)
Classes to Delete?: 1 (pending review)


Finalized Classes
	
Tested Classes
	SymbolPrivilege				(review documentation)
To Test
	SymbolPrivilegeLocal
	SymbolPrivilegeRegional
Review Classes
	SymbolSignature				(review before test)
	Symbol						(review before test)
	SymbolGate					(review before test)
	SymbolPath					(review before test)
	
Doing
	SymbolPathSet				(fix retain, remove, etc to use iterator)
	SymbolPathSetHashMap		(fix retain, remove, etc to use iterator)
	SymbolPathSetLinkedList		(fix retain, remove, etc to use iterator)
To Do
	SymbolResolve -> SymbolLinkable						(update)
	-	a symbol that can be linked to other symbols
	
	SymbolBase -> SymbolRegional						(update)
	
	SymbolScope											(update)
	SymbolType											(update)
	SymbolVariable					(each variable needs a set of paths that continue to become more strict)
									(each path leads to the same place, but they all grant different privileges)
									(normally, 3 paths, private/protected/public)
									(what happens when a variable is typecast?)
									(wrap the variable with another variable whose instance is the original variable, but it leads
									somewhere else)
									(Typecast extends Variable) (this doesn't seem right, but Typecast is almost identical to Variable)
		additionalPrivileges under restrictPrivileges
			restrictedPrivileges = intersect(providedPrivileges, union(additionalPrivileges, grantedPrivileges))
			
			useful for typecasting (may be protected or private privileges depending on type being cast to)
			add to SymbolGate
			
	SymbolFunction										(create)
	
	SymbolSignatureFunction								(update)
	SymbolSignatureTypecast								(update)
	SymbolSignatureFunctionOrTypecast					(update)
	-	first either resolve a function or resolve a type
	-		if resolved a type, then try to resolve typecast from type or fail
	
	SymbolQuery											(update)
	SymbolBaseQuery -> SymbolQuery (include)			(delete?)
	
	
	Module Pattern (vJASS)
	Inheritance Pattern
	Generic Pattern with an Implementation
	Function Overload Pattern
	Typecast Pattern
	Internal Pattern
	Closure Pattern
	Variable of same Type Pattern
	Variable Parameter of same Type Pattern
	Macro Pattern
	Define Pattern (parallel tables)

If anyone wants to review/touch the docs up on this, feel free ; D. Wrote 719 lines worth of test cases for this sucker.

package symboltable;

import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;

import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;

/**
 * A symbol privilege grants rights to access another symbol privilege. These rights are stored as
 * a set of flags. Accessing a symbol privilege requires the flags of that symbol privilege. The flags
 * provided are from the accessing symbol privilege.
 * <p>
 * A symbol privilege can intersect and union with collections. However, terminal nodes of these
 * collections must be symbol privileges.
 * <p>
 * A symbol privilege can be used for things like access modifiers or requirements
 * to go down a link in a virtual network. The default implementation dictates are
 * that all requirements are met if the required set of flags are a subset of the
 * provided set of flags.
 */
public class SymbolPrivilege implements Set<SymbolPrivilege>
{
	public static final ImmutableSet<SymbolPrivilege>													emptyMask		= ImmutableSet.of();
	public static final ImmutableMap<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>>	emptyMap		= ImmutableMap.of();
	public static final SymbolPrivilege																	emptyPrivilege	=
																															new SymbolPrivilege(
																																				emptyMap);

	/**
	 * A map from this point on is a Map containing sets, which in turn contain privileges.
	 * A mask from this point on is a Set containing privileges.
	 * The map organizes the privileges by class. Without this organization, privileges
	 * would not be able to interact with each other without having both the same types
	 * of privileges and the same order.
	 */
	private final ImmutableMap<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>>			map;
	public final boolean																				isComposite;

	/**
	 * This constructor is used when defining a new privilege.
	 * This new privilege is used in composite privileges.
	 */
	public SymbolPrivilege()
	{
		map = ImmutableMap.of(getClass(), ImmutableSet.of(this));
		isComposite = false;
	} // SymbolPrivilege

	/**
	 * This constructor is used by composite privileges.
	 */
	private SymbolPrivilege(ImmutableMap<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>> map)
	{
		this.map = map;
		isComposite = true;
	} // SymbolPrivilege

	/**
	 * This should be implemented by derived classes. This default implementation determines whether the provided
	 * privileges are accepted by the required privileges or not.
	 * 
	 * @param requiredMask		mask required for success
	 * @param providedMask		mask provided
	 * @return					provided mask was accepted by required mask 
	 */
	protected boolean acceptsMask(ImmutableSet<SymbolPrivilege> requiredMask, ImmutableSet<SymbolPrivilege> providedMask)
	{
		// the base privilege requires that provided privileges has all required
		// privileges
		return providedMask.containsAll(requiredMask);
	} // acceptsMask

	/**
	 * This can't be a general implementation as how deep the collections go for a map and the organization of
	 * those collections in that map is unknown.
	 * <p>
	 * Does not add empty masks to the map.
	 */
	private static <S extends SymbolPrivilege> void addMap(Map<Class<? extends S>, HashSet<S>> map,
															Map<Class<? extends S>, ? extends Collection<S>> mapToAdd)
	{
		HashSet<S> collection;

		for (Entry<Class<? extends S>, ? extends Collection<S>> entry : mapToAdd.entrySet())
		{
			if (!entry.getValue().isEmpty())
			{
				collection = map.getOrDefault(entry.getKey(), null);

				if (collection == null)
				{
					collection = new HashSet<S>();
					map.put(entry.getKey(), collection);
				} // if

				collection.addAll(entry.getValue());
			} // if
		} // for
	} // addMap

	/**
	 * Convert array of collections to a map.
	 */
	private static HashMap<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>> toMap(Object... collections)
	{
		// this is the final map
		HashMap<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>> map =
																					new HashMap<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>>();

		// if the object is a privilege, add it to the above map
		// if the object is some collection, convert it to a map and then add
		// that to the above map
		for (Object object : collections)
		{
			if (isClass(object))
			{
				addMap(map, ((SymbolPrivilege) object).map);
			} // if
			else if (Utils.isClass(Collection.class, object))
			{
				addMap(map, toMap(((Collection<?>) object).toArray()));
			} // else if
		} // for

		return map;
	} // toMap

	/**
	 * This is used to directly intersect an array of collections together into a map. A HashSet
	 * is used rather than an ImmutableSet because it is more efficient than recreating
	 * the structure over and over again. This HashSet is the map of the intersection.
	 */
	private static Map<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>> intersectToMap(Object... collections)
	{
		Map<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>> map = toMap(collections);

		// if the object is a privilege, retain that object's map
		// if the object is a collection, retain the intersection of its
		// elements
		for (Object object : collections)
		{
			// intersection with empty set is still empty
			if (map.isEmpty())
			{
				break;
			} // if

			if (isClass(object))
			{
				Utils.retainMap(map, ((SymbolPrivilege) object).map);
			} // if
			else if (Utils.isClass(Collection.class, object))
			{
				Utils.retainMap(map, intersectToMap(((Collection<?>) object).toArray()));
			} // else if
			else
			{
				map.clear();
			} // else
		} // for

		return map;
	} // intersectToMap

	/**
	 * This compiles a map into a SymbolPrivilege. Each set is first converted into an immutable
	 * set. After this, the map is converted into an immutable map.
	 */
	private static SymbolPrivilege compileMap(Map<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>> map)
	{
		// the passed in map will contain no empty masks

		if (map.isEmpty())
		{
			return emptyPrivilege;
		} // if

		// composite privileges must be greater than size 1
		if (map.size() == 1)
		{
			Set<SymbolPrivilege> mask = map.values().iterator().next();

			if (mask.size() == 1)
			{
				return mask.iterator().next();
			} // if
		} // if

		// have to create a new map to specifically hold ImmutableSet
		HashMap<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>> transitionMap =
																									new HashMap<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>>();

		// add the old map to the new map, converting Set to ImmutableSet
		for (Map.Entry<Class<? extends SymbolPrivilege>, HashSet<SymbolPrivilege>> entry : map.entrySet())
		{
			transitionMap.put(entry.getKey(), ImmutableSet.copyOf(entry.getValue()));
		} // for

		return new SymbolPrivilege(ImmutableMap.copyOf(transitionMap));
	} // compileMap

	/**
	 * @return				intersection of collections
	 */
	public static SymbolPrivilege intersect(Object... collections)
	{
		return compileMap(intersectToMap(collections));
	} // intersect

	/**
	 * @return				union of collections
	 */
	public static SymbolPrivilege union(Object... collections)
	{
		return compileMap(toMap(collections));
	} // union

	public static boolean isClass(Object object)
	{
		return Utils.isClass(SymbolPrivilege.class, object);
	} // isClass

	/**
	 * This will return true if the provided privileges are accepted by this
	 * privilege.
	 * 
	 * @param this				 required privileges
	 * @param providedPrivileges provided privileges
	 * @return					 provided privileges accepted?
	 */
	public boolean accepts(SymbolPrivilege providedPrivileges)
	{
		if (providedPrivileges == null)
		{
			providedPrivileges = emptyPrivilege;
		} // if

		ImmutableSet<SymbolPrivilege> providedMask;
		ImmutableSet<SymbolPrivilege> requiredMask;
		SymbolPrivilege derivedClass;

		// all masks must be accepted
		for (Map.Entry<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>> entry : this.map.entrySet())
		{
			providedMask = providedPrivileges.map.getOrDefault(entry.getKey(), emptyMask);
			requiredMask = entry.getValue();

			// the "acceptsMask" method is within a derived class
			// this will always be safe because compileMap excludes all empty
			// masks
			derivedClass = requiredMask.iterator().next();

			if (!derivedClass.acceptsMask(requiredMask, providedMask))
			{
				return false;
			} // if
		} // for

		// if all masks were accepted, then the privileges are accepted
		return true;
	} // accepts

	/**
	 * This will always return false for the empty set and composite sets. containsAll is more
	 * flexible.
	 */
	@Override
	public boolean contains(Object object)
	{
		if (isEmpty() && (this == object || object == null))
		{
			return true;
		} // if

		if (isClass(object))
		{
			return map.getOrDefault(object.getClass(), emptyMask).contains(object);
		} // if

		return false;
	} // contains

	/**
	 * contains all for composite, composite
	 */
	private boolean compositeComposite(SymbolPrivilege privilege)
	{
		for (Map.Entry<Class<? extends SymbolPrivilege>, ImmutableSet<SymbolPrivilege>> entry : privilege.map.entrySet())
		{
			if (!map.getOrDefault(entry.getKey(), emptyMask).containsAll(entry.getValue()))
			{
				return false;
			} // if
		} // for

		return true;
	} // compositeComposite

	/**
	 * contains all for defined, composite
	 */
	private boolean definedComposite(SymbolPrivilege privilege)
	{
		return privilege.isEmpty() || map.getOrDefault(privilege.getClass(), emptyMask).contains(privilege);
	} // definedComposite

	/**
	 * contains all for composite, defined
	 */
	private boolean compositeDefined(SymbolPrivilege privilege)
	{
		return map.getOrDefault(privilege.getClass(), emptyMask).contains(privilege);
	} // compositeDefined

	/**
	 * contains all for defined, defined
	 */
	private boolean definedDefined(SymbolPrivilege privilege)
	{
		return this == privilege;
	} // compositeDefined

	private boolean containsAllPrivilege(SymbolPrivilege privilege)
	{
		if (isComposite)
		{
			if (privilege.isComposite)
			{
				return compositeComposite(privilege);
			} // if
			else
			{
				return compositeDefined(privilege);
			} // else
		} // if
		else if (privilege.isComposite)
		{
			return definedComposite(privilege);
		} // else if
		else
		{
			return definedDefined(privilege);
		} // else
	} // containsAllPrivilege

	@Override
	public boolean containsAll(Collection<?> collection)
	{
		if (collection != null)
		{
			if (isClass(collection))
			{
				if (!containsAllPrivilege((SymbolPrivilege) collection))
				{
					return false;
				} // if
			} // if
			else
			{
				for (Object object : collection)
				{
					if (object != null)
					{
						if (Utils.isClass(Collection.class, object))
						{
							if (isClass(object))
							{
								if (!containsAllPrivilege((SymbolPrivilege) object))
								{
									return false;
								} // if
							} // if
							else if (!containsAll((Collection<?>) object))
							{
								return false;
							} // else
						} // if
						else
						{
							return false;
						} // else
					} // if
				} // for
			} // else
		} // if

		return true;
	} // containsAll

	@Override
	public boolean equals(Object object)
	{
		return (this == object || (isEmpty() && object == null))
				|| (isClass(object) && ((SymbolPrivilege) object).size() == size() && containsAllPrivilege((SymbolPrivilege) object));
	} // equals

	/**
	 * @return			map empty?
	 */
	@Override
	public boolean isEmpty()
	{
		return this == emptyPrivilege;
	} // isEmpty

	/**
	 * @return			privilege count
	 */
	@Override
	public int size()
	{
		int length = 0;

		for (ImmutableSet<SymbolPrivilege> mask : map.values())
		{
			length += mask.size();
		} // for

		return length;
	} // size

	/**
	 * @return			map to set
	 */
	private Set<SymbolPrivilege> toSet()
	{
		ImmutableSet.Builder<SymbolPrivilege> builder = ImmutableSet.builder();

		for (Set<SymbolPrivilege> mask : map.values())
		{
			builder.addAll(mask);
		} // for

		return builder.build();
	} // toSet

	/**
	 * @return			map iterator
	 */
	@Override
	public Iterator<SymbolPrivilege> iterator()
	{
		return toSet().iterator();
	} // iterator

	/**
	 * @return			map to array
	 */
	@Override
	public Object[] toArray()
	{
		return toSet().toArray();
	} // toArray

	/**
	 * @return			map to array
	 */
	@Override
	public <T> T[] toArray(T[] a)
	{
		return toSet().toArray(a);
	} // toArray

	/**
	 * Not supported.
	 */
	@Override
	public boolean add(SymbolPrivilege e)
	{
		throw new UnsupportedOperationException();
	} // add

	/**
	 * Not supported.
	 */
	@Override
	public boolean remove(Object o)
	{
		throw new UnsupportedOperationException();
	} // remove

	/**
	 * Not supported.
	 */
	@Override
	public boolean addAll(Collection<? extends SymbolPrivilege> c)
	{
		throw new UnsupportedOperationException();
	} // addAll

	/**
	 * Not supported.
	 */
	@Override
	public boolean removeAll(Collection<?> c)
	{
		throw new UnsupportedOperationException();
	} // removeAll

	/**
	 * Not supported.
	 */
	@Override
	public boolean retainAll(Collection<?> c)
	{
		throw new UnsupportedOperationException();
	} // retainAll

	/**
	 * Not supported.
	 */
	@Override
	public void clear()
	{
		throw new UnsupportedOperationException();
	} // clear
} // SymbolPrivilege

 

[Ruke]

I'm talking to you Ruke >: o

i solved what i need in only one class

 

[Nestharus]

Ok everyone, so there are updates to the design.


1. Originally, I was using Symbols, Gates, Links, and Privileges. Privileges would be provided by links and revoked by Gates. This made everything kind of complex so I'm changing things around.

Every scope has its own privileges, depending on what that scope can entail. Certain links will grant privileges from what it points to. This means that when you nest scopes, the nested scope can inherit the privileges of the parent scope. It'll also let you do special types of links with vJASS modules (inherit privileges from both the scope where the module is declared and the scope where the module is implemented). Privileges are never revoked. Since everything has its own set of privileges (public excluded), they can just run around with that set and access what they will. Less overhead as well considering that it'll be using a HashSet for practically O(1) lookup and that the privileges won't have to be changed as you move through gates.

2. Parallel processing for compilation.

Step 1: Definition Phase

Whenever the main thread comes across a scope, it will assign it to another thread. The main thread will then skip to another scope. The assigned thread will define everything for that scope and create linking tasks and semantic tasks that are to be performed later.

Step 2: Linking Phase

All scopes have been defined, so now they are linked together. Each thread has a buffer of tasks to be performed. The main thread will fill up the buffers of each other thread and then continue to add to them.

Step 3: Resolution Phase

Same as above, but for things like semantic analysis.


After this, we have optimization and translation and stuff. Haven't figure out those pieces yet. For now, I want to do steps 1-3. This should make the compiler run extremely quickly. In the worst case, it'll run in O(n). Certain special files, if every line is guaranteed to contain a single definition, like the native file, can divy the lines up as tasks. Could actually do an initial scan and figure out what are scopes and what are definitions. One thread can then work at defining everything for the current scope while the other threads work out the other scopes. We could actually just have the main thread do this ; ), and then as it comes across other scopes, define them and hand them out.


The idea with all of this is to make sure that no thread is accessing or changing the same data at the same time except for the buffers, which can be modified atomically. No locks will then be required ^_-. As long as we only ever have consumer-producer relationships in 1:1, this'll work without locks. Hopefully this will prove to be the fastest compiler to date : ).

 

[Nestharus]

So, here are the updates.

1. The current grammar is revoked and it'll just be vJASS, except for 2 things that are pretty important. _ will be replaced with . and : will be removed. There are some extra features and stuff under consideration, but they won't be allowed in until they are approved by the community. The _ and : are too important. With them in, they break everything.

2. The project is on hold until we can get 15 people total to volunteer to work on it. If there isn't enough interest in it, there's no point in doing it.


The project don't only need coders. We need people to code, write documentation, and review/approve possible syntax to be introduced into the compiler. If you're interested in contributing to the project, I am sure that there is something you can do : ). However, we are going to want atleast 7 coders.

For syntax approval, everything that gets past the initial committee can be put up as a poll and voted upon over the course of 1 week : ).


This project is going to go nowhere unless it gets support from the community, so if you are interested in fixing all vJASS problems, making the compiler faster, and introducing a slew of new features, this project is of benefit to you. We're talking features like generics (List<Type>), new method operators (delegate method operator, typecast method operator), and even possible ways to download/install resources automatically that are required (lots of reqs will no longer be detrimental). There are all sorts of things that can be done, but none of it's going to happen without support from the community : ).


If you're interested in helping out, please reply. If we get 15 people, we can ask Ralle for our own forum and start working on this thing.


Anyways, here are current ideas from me. None of this has been approved. It's up to the "committee" and the polls to decide what gets approved.

struct S1 extends array
    public integer b
endstruct

struct S2 extends array
    public method operator delegate a takes nothing returns S1
        return this
    endmethod

    public method operator typecast b takes unit u returns thistype
        return GetUnitUserData(u)
    endmethod
endstruct

struct S3<T>
    private T data

    public method store takes T value returns nothing
        set data = value
    endmethod
endstruct

// see how T is passed in, but it could be a type or anything too
struct S4<T, T2> extends S3<T>
endstruct

// when wanting a specific macro implementation for array space
struct S5 extends S3<unit>
endstruct

function Test takes nothing returns nothing
    local S2 s2 = 0
    local unit u = ...

    // when wanting to go off of a shared macro implementation
    local S4<unit, unit> s4 = S4<unit, unit>.create()

    set s2.b = 5 // S1(0).b = 5
    set S2(u).b = 5 // S1(GetUnitUserData(u)).b = 5

    call s4.store(u)
endfunction

// idea to use existing allocators with overload, but drop the polymorphism in inheritance
static struct M
endstruct

// idea for one-time code execution
integer h = 5
set h = 6

Here are topics for consideration-

local unit u = ...
set u.life = 60 // how will this be supported?

// generics for modules
module S<T>
    public method boo takes T t returns nothing
    endmethod
endmodule

struct M
    implement S<unit>

    // masking module members like with delegates
    public method boo takes unit t returns nothing
        call Module.S.boo(t)
    endmethod

    // function overloading
    public method boo takes integer t returns nothing
    endmethod
endstruct

Nested textmacros
Private scopes
Scopes within structs
Automatic grouped code ordering based on scope/library usage (use a function from a scope and that scope is done before you)
Automatic local code ordering (use a function in current scope and that function is placed before you)

function M takes nothing returns nothing
    local integer i // var 1

    if (true) then
        local integer i // masks i, var 2
    endif

    if (true) then
        local integer i // masks 2, var 2 again, not a third local var!
    endif
endfunction

Use # or //! for preprocessor directives
AST for preprocessor
Multiple import of a given file (use preprocessor ifs like in c++)
Preprocessor loops
Preprocessor functions with way to dump strings to script
String interpolation ($, ${}?) (print("hello $name!")
Script interpolation (between preprocessor and script, but how? also, how to access single elements or short blocks of preprocessor script and vice versa like in strings?)
Regex support (how will this be done? When the compiler comes across an existing regex expression in a string, it should replace it with the correct stuff, but how to define a regex expression?)
Regex comparison (how? Perl uses =~)
Smart polymorphism (binary ifs vs trigger evals, switch between them as binary ifs become too slow)

Expanded array operators

method operator [][][] takes integer i, integer i2, integer i3 returns integer
endmethod

integer m = thistype[1][2][3] // use the above

special inline keyword
lambda expressions
function pointers (how?)
protected access modifier (accessible by struct/inherited structs or by members of same scope in multi-scope)
internal access modifier (accessible by file/trigger)
multi scope declaration (private is local to 1 scope declaration, protected is across all)
object generation/manipulation with object table (how? talking object editor)
a way to avoid op limit for large operations that keeps code readable (see things like BigInt)

it'd also be nice to have a way to declare a table struct instead of an array struct

 

[SA Dashie]

Seems like some pretty cool stuff Nes.
Hope for the best.

One thing especially that would help me is those nested text macros you mentioned.

 

[edo494]

// generics for modules
module S<T>
    public method boo takes T t returns nothing
    endmethod
endmodule

struct M
    implement S<unit>
    implement S<widget>

    // masking module members like with delegates
    public method boo takes unit t returns nothing
        call Module.S.boo(t)  //@ambiguous@
    endmethod

    // function overloading
    public method boo takes integer t returns nothing
    endmethod
endstruct

So yeah

 

[btdonald]

Pure awesomeness ... Why always you, Nes ))))

 

[Waffle]

btw this IS a call to arms.. so if u actually care ehm.. u know..

 

[Nestharus]

Here's the latest news on the Compiler. Waffle and I have re-engineered the base library into something called Symbol. It is now much simpler and smaller.

The compiler will have 2 projects. The first project is Symbol, which is a library for creating symbol tables. The second project will have the parser, lexer, symbol table, etc. It'll be the concrete implementation.

We are using Gradle, JUnit, GitHub, Guava, and ANTLR4. We're also likely going to use something to generate UML diagrams from the code.

We are still looking for people to contribute. Writing test cases, writing documentation, working on diagrams, working on syntax, solving syntax problems, and working on compiler design are all very important. We need lots of people.

The plans for tomorrow are that we will work on milestones. That is, we're going to lay out what we need to do for the project and the phases we're going to do everything in.

Please, if this project seems like it'll be of benefit for you, contribute : ). We definitely have something you can do. If we get enough people and the project progresses well, we can probably totally ask Ralle or our own forum too.


Here are some other cool things.

1. Symbol can be used for any compiler. Want to make your own language? Want to make translators? Galaxy? : )

2. The symbol table will be largely reusable for most languages because it is built on Symbol. This means that a lot of the work is done for you.

3. The Symbol Table population is largely reusable (the model atleast).

4. The optimization is largely reusable as it will be for an intermediate language.


So if you want to work on something different, like a cool compiler for Starcarft 2 Galaxy, you can totally contribute to this project and get a lot of out it.


This project will fail if it's just the two of us. Waffle only has a month and I don't have much time. We definitely need people. If you want this project to actually succeed, help out.

 

[Nestharus]

Daily Progress Log:

We've finished Symbol. Well, actually, Waffle's going to be running some unit tests on Symbol to make sure everything's good.

What's next?...

1. JASS Grammar
2. Concrete Symbol Implementation for JASS
3. Parse Tree Listener
4. Symbol Table Creation (as in, populate, link, all of that jazz)
5. Intermediate Language Translator
- Optimizer Goes Here
6. JASS Translator

So, we'll be translating from... JASS to JASS! : D

After that, we'll slowly add in features and stuff ^_-.

There's still TONS of work to do and we need you : ). If you want to be part of something special, let us know.

I'll be sure to continue to post a log when we finish something =). Tomorrow I kind of want to generate all of the diagrams too and put them up on GitHub.

Oh.. and without further ado

https://github.com/nestharus/Symbol

And here is the actual Symbol class. Note that the package needs to change and that Signature should probably be renamed to SymbolSignature : o. The below magical class can handle all of our crazy features, from generics to whatever.

The current debate is whether phases, getResolutionPhases, and the logic of getClosestSymbol should be moved into a new class called ResolutionTable. ResolutionTable would have the getClosestSymbol in the resolve method. I don't think that resolution logic belongs in the Symbol class.

package org.gradle;

import java.util.Arrays;

public class Symbol<T extends Enum<T>> {
	private final ResolutionPhase<T>[] phases;
	private final Signature signature;
	private final Privilege requiredPrivileges;

	@SafeVarargs
	public Symbol(Privilege requiredPrivileges, Signature signature,
			ResolutionPhase<T>... phases) {
		this.requiredPrivileges = requiredPrivileges;
		this.signature = signature;
		this.phases = phases;
	}

	@Override
	public boolean equals(Object object) {
		return object != null && object instanceof Symbol
				&& signature.equals(((Symbol<?>) object).signature);
	}

	public Symbol<T> getClosestSymbol(Privilege providedPrivileges,
			Signature signature) {
		Symbol<T> closestMatch = null;
		int closestDistance = Signature.INFINITY;
		int currentDistance = 0;

		for (ResolutionPhase<T> phase : phases) {
			for (Symbol<T> symbol : phase.getSymbols(providedPrivileges,
					signature)) {
				currentDistance = signature.distance(symbol.getSignature());

				if (currentDistance < closestDistance) {
					if (currentDistance == 0) {
						return symbol;
					}

					closestMatch = symbol;
					closestDistance = currentDistance;
				}
			}
		}

		return closestMatch;
	}

	public Privilege getRequiredPrivileges() {
		return requiredPrivileges;
	}

	public ResolutionPhase<T>[] getResolutionPhases() {
		return phases;
	}

	public ResolutionPhase<T>[] getResolutionPhases(
			@SuppressWarnings("unchecked") T... phaseTypes) {
		Arrays.sort(phaseTypes);

		@SuppressWarnings("unchecked")
		ResolutionPhase<T>[] phases = new ResolutionPhase[phaseTypes.length];

		for (int i = 0; i < phases.length; ++i) {
			phases[i] = this.phases[phaseTypes[i].ordinal()];
		}

		return phases;
	}

	public Signature getSignature() {
		return signature;
	}

	@Override
	public int hashCode() {
		return signature.hashCode();
	}
}

 

[Empirean]

Intellisense pls

 

[Nestharus]

Intellisense isn't part of a compiler : ). That's the IDE.

 

[Empirean]

Sorry, never tried making a compiler before.