[Snippet] Scrambler

[Nestharus]

This resource has moved here.

--

gone

 

[PurgeandFire]

It looks awesome. I can't see anything wrong with it. Simple to use too.

 

[Nestharus]

Fixed a bug and removed a useless array ;D.

 

[Bribe]

Seeing as how this is used for protection, having these short, unreadable variable names gives it extra safety so no one can reverse the process. Save/load codes will make good use of this. Approved.

 

[Nestharus]

Updated-

Added salt to hash.

 

[Magtheridon96]

Updated-

Added salt to hash.

Brilliant
Now cracking the codes on Encoder is closer to impossible ^^ (It was already nearly impossible anyway )

 

[Nestharus]

Eh, I'm just implementing new security that was developed when working on a high security save/load code for Bond009's footmen wars map =).

I posted a thread on it in World Editor Help Zone if you'd like to read up ; P


Yes, Encoder 3 here we come ; D

 

[Magtheridon96]

Yes, Encoder 3 here we come ; D

Please Don't change the API if possible D:

 

[Nestharus]

Please Don't change the API if possible D:

Sadly, API is changing.

I'll still support Encoder 2 though =).

but only Encoder.create will change, so no worries

 

[Magtheridon96]

Sadly, API is changing.

I'll still support Encoder 2 though =).

but only Encoder.create will change, so no worries

^^

I read the thread you posted in the World Editor Help Zone.
Encoder 3 sounds really promising

I have another suggestion for it too:
The number of shuffles could depend on the name of the player:
Shuffles: StringHash(player_name + base)%4 + 2

Basicly, if the entire encoding procedure is player unique, the security would be much better.

 

[Nestharus]

At the moment, I'm strengthening the shuffling algorithm because right now it semi sucks D:.

Also, I don't know about basing the number of shuffles off of name of player as that wouldn't really strengthen it at all. Each player gets a different key, so by giving them different amounts of shuffles, you end up making collisions if that makes sense =P.

 

[Magtheridon96]

Also, I don't know about basing the number of shuffles off of name of player as that wouldn't really strengthen it at all. Each player gets a different key, so by giving them different amounts of shuffles, you end up making collisions if that makes sense =P

lol
I didn't realize that
Forget shuffles xP

... Maybe something involving prime numbers may work
I don't know exactly how, but I recall something about Prime Numbers involving some sort of encryption algorithm.. I'll be right back after I do some research =P

 

[Nestharus]

Yea, check this out =)

Name 1		Name 2		Shuffles
43521		45213		1
53214		25143		2
12435		43521		3
42351		53241		4
53124		41352		5
13245		31542		6

Name 1		Name 2		Shuffles
43521		XXXXX		1
XXXXX		43521		3

 

[Magtheridon96]

lol

I don't see a pattern to that

edit
While playing with some prime numbers, i found something:

1,2,3,5,7,11,13,17,19,23,29,31,37,41
1,1,2,2,4,2,4,2,4,6,2,6,4

I can't believe I didn't notice this before xD
Maybe you could generate the key using the sequence of the intervals between the occurrences of prime numbers consecutively ^^
I'll try to see if collisions are possible

edit
lol
I continued the sequence, but I found some collisions with 3 digit keys

 

[Nestharus]

Ahem, for the different shuffles, you have 2 different keys and shuffle 1 is the same as shuffle 3. However, on each level, the shuffles are different. That's why changing shuffle based on player name is a very bad idea =P.


Apparently this doesn't work well the closer a number is to binary powers of 2... gaaah


If the number is perfect 2^power, then it won't shuffle at all regardless of the key...

 

[Magtheridon96]

lol
Then forget it

 

[Nestharus]

Well, I'm working at fixing this atm =)

 

[Magtheridon96]

This may help:

library Powers
    function isPowerX takes real i, integer j returns boolean
        loop
            exitwhen i<j
            set i=i/j
        endloop
        return i==1.0
    endfunction
endlibrary

It's as fast as possible ^^ (i think..)

 

[Nestharus]

that will help very little


I figured out how to fix this tho ; ).

Shuffle by prime bases, one after the other =D

shuffle(2)
shuffle(3)
shuffle(7)
shuffle(13)
shuffle(17)
shuffle(19)
shuffle(23)
shuffle(29)
shuffle(31)
shuffle(37)
shuffle(41)

^)^

 

[Magtheridon96]

D:< You forgot shuffle(5) xD

edit
Brilliant idea =D

Btw, you also forgot shuffle(11) ^)^

 

[Nestharus]

The update is coming along nicely.

Here is the latest shuffling algorithm results for numbers 65536 to 65615

Value	Scrambled	Binary		Runs			
65536	99655		0		1	Mean =            	219091.975
65537	81055		0		1	R =                  	38.000
65538	398993		1		2	n0 =                 	52.000
65539	418487		1		2	n1 =                 	28.000
65540	81902		0		3	n =                 	80.000
65541	20608		0		3	E( R) =             	37.400
65542	367578		1		4	Var(R ) =         	16.311
65543	164939		0		5	StDev(R ) =     	4.039
65544	56919		0		5	Z =                  	0.149
65545	288126		1		6	p-value =        	0.441
65546	246815		1		6	Randomness      	88.19%
65547	67598		0		7		
65548	20466		0		7		
65549	134593		0		7		
65550	341920		1		8		
65551	109513		0		9		
65552	33806		0		9		
65553	286742		1		10		
65554	57966		0		11		
65555	20076		0		11		
65556	338831		1		12		
65557	237383		1		12		
65558	37565		0		13		
65559	86806		0		13		
65560	128358		0		13		
65561	10556		0		13		
65562	18562		0		13		
65563	105933		0		13		
65564	64761		0		13		
65565	99623		0		13		
65566	76262		0		13		
65567	49511		0		13		
65568	150022		0		13		
65569	8721		0		13		
65570	401983		1		14		
65571	579845		1		14		
65572	176988		0		15		
65573	139061		0		15		
65574	83663		0		15		
65575	13286		0		15		
65576	52993		0		15		
65577	781222		1		16		
65578	396229		1		16		
65579	304509		1		16		
65580	35788		0		17		
65581	406664		1		18		
65582	61193		0		19		
65583	55323		0		19		
65584	520326		1		20		
65585	140189		0		21		
65586	111435		0		21		
65587	78679		0		21		
65588	171954		0		21		
65589	32572		0		21		
65590	154348		0		21		
65591	253470		1		22		
65592	117766		0		23		
65593	211644		0		23		
65594	486599		1		24		
65595	90875		0		25		
65596	862767		1		26		
65597	107701		0		27		
65598	80581		0		27		
65599	1212346		1		28		
65600	55392		0		29		
65601	416560		1		30		
65602	383168		1		30		
65603	261443		1		30		
65604	324677		1		30		
65605	97915		0		31		
65606	129520		0		31		
65607	274502		1		32		
65608	132315		0		33		
65609	158255		0		33		
65610	360085		1		34		
65611	98331		0		35		
65612	722671		1		36		
65613	120352		0		37		
65614	204130		0		37		
65615	955392		1		38

A P-Value of .5 is about as random as it gets ; ).


This took the idea of switching between different primes rather than working up towards the higher primes.

I'm still working it out, but my next idea should be the final algorithm used.

Also, the amount of scrambles used here was 3. I'm going to try for 1 scramble with a P-Value close to .5

Previous algorithms-
P-Value .02
P-Value 0.11867


So you can see, I've been slowly getting better >: )

 

[Bribe]

It's as fast as possible ^^

It's really not as ModuloInteger does the exact same thing but with O(1) complexity instead of a loop.

 

[Magtheridon96]

A P-Value of .5 is about as random as it gets ; ).


This took the idea of switching between different primes rather than working up towards the higher primes.

I'm still working it out, but my next idea should be the final algorithm used.

Also, the amount of scrambles used here was 3. I'm going to try for 1 scramble with a P-Value close to .5

Previous algorithms-
P-Value .02
P-Value 0.11867


So you can see, I've been slowly getting better >: )

haha >
I knew prime numbers could be involved in encryption >
Those results are really interesting ^^
And to be exact, you're p-value is 0.44095 >

 

[Nestharus]

Here is the latest data on the new version-
http://www.hiveworkshop.com/forums/pastebin.php?id=fk1keq

I've just been testing it... yea, it's been up and ready to go all night =P.


Anyways, I think I'm pleased with the data so I'll do some final tests tomorrow to make sure everything is working and then I'll put it up =).


edit
As the data set increases in value or decreases in value, the average increases/decreases in value. Because of this, lower values are weighted on one side and higher values are weighted on another side with medium values in the center. This is the reason why this fails the runs test with large numbers of data.

This behavior is to be expected and is actually desirable for this =).

Within a local area of 20 or 80, the data is very even as the data set isn't large enough to cause this effect.

So given a number, you can expect it to fall within a specific range. The larger the number, the bigger the range it can fall in. As save/load codes are very large, the effect that takes away from the randomness will take much longer to occur (I was dealing with a very small value of 65536), meaning that this will pretty much generate a random number given a number.

To give you an idea of the range for larger numbers, 65536 goes between around 4500 and 1.2 million.

 

[Magtheridon96]

Encoder 3.0.0.0 by tomorrow??

 

[Nestharus]

Probably =).

I spent all day gathering statistics on this =P.

Oh, would you like to see the new settings area for Scrambler? You can define your own scrambling algorithms using 5 prime bases =D.

The below algorithm is the one I've been using for these tests ; P.

************************************************************************************
*
*   SETTINGS
*
    globals
    /*************************************************************************************
    *
    *                                       SALT
    *
    *   Refers to what to append to the player's name when generating scrambler keys
    *   for each player. This is essentially like a password. It's impossible for another
    *   map to descramble a number without this password.
    *
    *   This password can be any string, such as "7/ah+53~r\\ZZ"
    *
    *************************************************************************************/
        private constant string SALT = ""
    endglobals
    
    /*************************************************************************************
    *
    *                                   SetShuffleOrder
    *
    *   Creates the shuffling algorithm using 5 different prime bases.
    *
    *   Shuffling mixes a number up using a variety of bases to produce results rivaling
    *   top random number generators.
    *
    *   Different bases ensure better mixes.
    *
    *   Bases: 2,3,5,7,11
    *   Shuffle Array: so
    *
    *   Ex:
    *
    *       set so[0]=3         //first mix
    *       set so[1]=2         //second mix
    *       set so[2]=7         //third mix
    *       set so[3]=2         //fourth mix
    *
    *       return 4            //return number of mixes
    *
    *************************************************************************************/
    private keyword so
    private function SetShuffleOrder takes nothing returns integer
        /*************************************************************************************
        *
        *                                       MIXES
        *
        *   array: so
        *   bases: 2,3,5,7,11
        *
        *************************************************************************************/
        set so[0]=5
        set so[1]=2
        set so[2]=3
        set so[3]=11
        set so[4]=2
        set so[5]=7
        set so[6]=3
        
        /*************************************************************************************
        *
        *                                       MIX COUNT
        *
        *************************************************************************************/
        return 7
    endfunction
*
************************************************************************************

 

[Magtheridon96]

So we should change the "so" array sequence to something like:
41
83
37
17
11

Or could we use prime numbers like 499 or 5081?


edit
I think we just coined the term: Prime Base Encryption

 

[Nestharus]

Well, it does give a list of the bases you can use >.>

 

[Magtheridon96]

Well, it does give a list of the bases you can use >.>

I knew that, but what I didn't know was that we're limited to them... =P It's not a problem anyway

 

[Nestharus]

Ok, I solved the strange output on this..

Why do numbers around 65000 have output between 4000 and 1.2 million?

Going above 65536, the bits in a given number is 17.

When a number is shuffled in something like base 3, 1.5 bits are moved around for every movement. Every digit in base 3 is made up of 1.5 binary digits.

In base 3: 2
In base 2: 10

In base 5: 4
In base 2: 100

In base 7: 6
In base 2: 110

In base 11: A
In base 2: 1011


If you look at base 11, it goes as small as .25. In base 7, it's 1/3. In base 5, it's 2/3. None prime bases would just be a combination of base movements in every movement. For example, 4 would be 2 binary movements for every movement.

So these different fractions split apart and combine binary digits, thus giving the absolutely wild ranges that can be seen coming from Scrambler.

This makes reading a number w/o knowing the algorithm used to mix it up totally impossible.

Recall that this algorithm both includes that bases used in every shuffle as well as the hash of the player's name + a string value.

 

[Magtheridon96]

This makes reading a number w/o knowing the algorithm used to mix it up totally impossible.

Recall that this algorithm both includes that bases used in every shuffle as well as the hash of the player's name + a string value.

Best news I've heard all day =)

Btw, Instead of Base 62, you could use Base 80 as a default Base =P
I'm sure everyone has access to these characters:

"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890!@#$%^&*()=+[]<>{}"

Ofcourse, that would limit the delimiters to: ":" or ";" or "-" , but honestly, who cares

edit
Nevermind.
I just read the argument you and Bribe were having over a similar idea xD

 

[Nestharus]

Updated to Scrambler 3

Tests going over larger numbers in different scenarios to see if any patterns arose. None found >: D.

Base 64=	<23456789abcdefgh>+kmn=pqrs^uvwxyzABCDEFGHKLMNOPQRSTUVWXYZ@#?&%$	
Base 63=	<23456789abcdefgh>+kmn=pqrs^uvwxyzABCDEFGHKLMNOPQRSTUVWXYZ@#?&%	
2^50=	1125899906842624	
Prime=	2736653538294121	
		
64 2^50  	64 Prime	63 Prime
		
5<<<<<<<<	aOgBf2xDH	c2P%VhA2G
		
78BWPx4ak	8QvfC6w0=	9SkYRdE=Q
9ZW=HQkA?	87cb<Fe#X	947HQ^wHc
bzv3sagrd	DL6T+LaNk	KO6GE%hh+
7<ckQ5NvA	ghDcDxVC2	>kq#u<FeQ
aXX>gPEf3	hXfmUS^fW	k9<5%8%YS
7VfZ^+pp<	eb9qPFDMr	f@>s%r#d@
cy+exR<Vd	dUgsSYYsE	fzUFx8&Zh
b9dq=wZm9	7WXeu@#=h	8R<EbfxW#
cMWgrSMCa	qMfGu<>Uq	u<qY%?dGW
6>cbzR$ZK	cVCFmTk5Y	esW=DDggD
6e5KvR=cc	6A=EM&3@E	7>H23Fe>S
ecx@xfZaN	fSUkNB9E>	hRb+#WKgs
9d<d<Rrkr	d^=hvCZxU	f73CZ3MTw
dZZ#YZGVY	dAg#kdLTv	feP2xDZ^=
6Q9bC?v43	h@frwy8KR	kc=cMXxcn
gMsnqqYHO	6wKaaVf8>	7ez8KNup?
9qdE$qD?7	exY3&8YW+	gkA%#u5x0
9yb>PAnzB	cFFg#ewOs	ec>v7OzPs
d%sRLp<B2	+p&uf#rcu	mV<NF3uEg
9cDXCXaQq	gBZdwbb5m	>G?^%^z%5
7hm=GHHfa	8UU4POXV+	9Xc=@vaM+
h>@FQTn>Z	v%98dEk8%	z%K^=X5fX
h6&$Ugb&b	z6fNH9%rn	Dy&SO#m53
9a?f%h5?4	8da^GrN?D	9aQT0>^rb
bHAngfKHd	5KQCAykzL	6+Cqs8hwF
b%&AVyAd?	>L%qWZ+AR	m4YMpD?6L
9KuBGS#4=	7gSvhG736	86q33fYKL
 +FxxpdyUv	r4=v7?zqf	usepKf7F^
aHxgyVUT+	yDPW<4bY=	C?UhP>Z=a
5ey=qZ@ED	9UzLQ<V@e	b<m5N<+5Y
>T8hF#svM	bVR#p@aY<	d+G6mrC%3
k9S$bT8&g	cNwnSbS$G	e>TNGAKg6
9g+<phf4U	xUyFMrZFH	C6VP?SB=N
 +qYLsue9h	C60^LBKGy	G@ZqvCF6W
5UCc86Z+5	k>y<6y>q$	nWheVMbbm
aQT=RAm6G	c%uUZF4BN	eCQXspa+>
7sap#gBNf	9QkEp<WD9	a@EfeUq%0
dGzsYGqZ%	>KmKc%S8<	m34xnP4v%
dAmXak7S8	d3aVKvsSR	eGZpXzAAm
7w3VvCTsF	fn7536Vzm	hg&A^8KEy
6>=UEw@WP	hkwFcMg@T	 +x6Hspq=x
>Ym$xV$md	cSXCBb?3$	epVbyU^Q8
f&SNY66TY	7SGgrg8fh	8M>R=BK7%
bGn7$vrVq	9ERPfXAFy	aP?@vG9@2
c?A2xq90G	DsYPF%<uw	K^%TuW@Eb
ad=AFceY4	eG+fTqCR4	gu#yDV^>#
gUgf4FWk+	x2ncBXm7D	BcM054mAu
6&65nH=$$	=wN$Vv$fT	rxqQM7>vX
99FkycbF8	hkcx+q63X	 +wPGh76bX
aQT?AD%b@	cdc+efzZg	dLGwz7@L@

 

[Magtheridon96]

All I have to say is WOW..

That settles it i think.
I would usually expect something like 5<<<<<<<< to create some sort of patern
The fact that it didn't shows how great this encryption is
Did you run tests for lower values? (1-10000)

You are going to credit me right?
After all, I did come up with the idea of using prime numbers here

 

[Nestharus]

After all, I did come up with the idea of using prime numbers here

No you didn't... the idea I came up with was entirely different from yours >.>. I thought of how base 3 numbers were 1.5 bits, and then I picked prime numbers because that's what I always pick for cryptography : p

I ran the first test with base 3 to base 2 to see how well it worked, then I gathered the statistics.

You contributed nothing, so why should I credit you : P

 

[Magtheridon96]

No you didn't... the idea I came up with was entirely different from yours >.>. I thought of how base 3 numbers were 1.5 bits, and then I picked prime numbers because that's what I always pick for cryptography : p

I ran the first test with base 3 to base 2 to see how well it worked, then I gathered the statistics.

You contributed nothing, so why should I credit you : P

..Yeah you're probably right =P
All I did was mention the word prime xD

Back to the actual topic;
I'm going to test this for lower values to see if a pattern is found

 

[Nestharus]

Lower values won't work very well because there are less bits to scramble.

Keep in mind that even 65536 in save/load is an unrealistic number because you are generally going to have atleast a 6 digit checksum.


But I am curious about the lower numbers =)

here
http://www.onlineocr.net/default.aspx


that's how i ripped results out. I did 150 results at a time with 10 per line =P, then I put it into excel ;D. Well, that was for the statistics Oo.

The only pattern in this is that as the numbers grow, so does the average of the output, and we are talking 500,000 from 100 samples to 990,000 with 1200 samples ;P.

 

[Magtheridon96]

I did some tests with lower values.

Base 88:    0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ<>!@#$%^&*()-=+_~[]{}?/`;:

1	13	169	453

1	1Y	X@	59=
2	28	A)	6:{
3	@	P@	5[u
4	1l	lkx	b54
5	1~	15;	6:b
6	+	I>	7rG
7	1k	II.<	2s0
8	1%	)v	2`9
9	)	%>	32e
a	(	10(	2:/
b	1w	1Yh	7t7
c	1c	lb/	d[q
d	1F	30q	1pi
e	18	>f	5r*
f	~	20%	9&N
g	1h	2A6	3]}
o	13	2e[	3[5
i	1C	10u	3EK
q	-	10	2v1
h	1A	1KQ	3T@
r	14	r9	7ZV
j	1r	rN	7X]
p	2h	Q%	6cG
k	1n	%I	2I1
u	1*	11B	6xr
m	1+	Zc	4RR
s	1G	%v	72D
l	_	)$	6t;
w	1d	Ne	50a
n	17	IR	4W^
v	1[	24	3pR
K	=	A<	3EY
y	1x	<;	3Ga
T	1W	lic	3Yk
D	}	1#n	1^!
x	1v	ljn	47W
Q	1M	Iwc	3UV
G	2P	_X	3+p
Z	1^	>E	3!w
E	1V	ljt	3_t
N	1i	4h<	434
J	1?	KS	3t!
U	1g	1Nk	2;y
W	1#	Ivj	7GG
C	1K	?s	5N:
V	2q	w{	d1`
P	1y	%o	5sT
t	2u	Ox	3x5
Y	1E	){	5N%
O	2I	$@	c0%

Conclusion: The larger the number, the greater the random factor

edit
I ran 2 more tests and thus, my statement is confirmed

 

[Nestharus]

I already knew that =P.

Also, the first bit doesn't shuffle in with the rest, so you have to have at least like 3 bits to even do anything =).


Now, if you had at least 100, it'd prob shuffle nicely =).

edit
The primary thing I was curious about was how different lower numbers would be =P

 

[BBQ]

This is awesome and everything, yet, I can still crack your Encoder rather easily. Does that make me even more awesome?

 

[Nestharus]

Haven't updated Encoder to use this yet =).

Tell me if you can crack it w/o looking at the map once Encoder 3 is out ; D.

 

[BBQ]

Haven't updated Encoder to use this yet =).

Tell me if you can crack it w/o looking at the map once Encoder 3 is out ; D.

You fail to see that any WC3 save/load system that is available to the public is as easy as a breeze to crack.

 

[Nestharus]

Well, if you opened the map and retrieved all of the settings (requiring you to be familiar with all of the systems), then yea, you could crack it.


W/o opening the map, even if you know all of the systems used, w/o the settings, it's impossible w/o like trillions of brute forces. You'd need to crack everything at once.

There are 2 levels, everything and then the shuffling. You can't crack the first level w/o cracking shuffling and you can't crack shuffling w/o cracking the first level. Because every value is made up of every other value in a number when using shuffling, a brute force crack would just be unfathomable >.<. You'd have to brute force in this manner in order to crack

descramble w/ a random scrambling algorithm + key (loop through every possible key + algorithm)
read using a random checksum + random player salt checksum value + random base + random value sizes + random value orders + random values (loop through everything possible)
change 1 value
reapply all security
see if code works

If you can manage like 10 correct values in a row, then you probably cracked it. This is why I say that with the scrambling and even minor underlying security, cracking is impossible =P.

edit
Updated to 3.0.0.1

This update just increases the speed of Scrambler.

I had too many trigger evals >: P

 

[Nestharus]

Optimized comparisons and removed extra space

 

[Bribe]

Opening thread

 

[Nestharus]

Updated Scrambler to work with BigInt v2

 

[jim7777]

ohh BigInt v2 plus a new scrambler, oh well

still, as of now, I can't decode these stuffs using php

it's killing me D:

 

[Nestharus]

To decode the stuff using all of the settings, you'll need to figure out wc3's string hashing.

 

[jim7777]

yeah..

perhaps i might just construct my own scrambler-like system..

at least the website named "jade" (i don't know the exact name, can't find it on google too) which will just decode a save code and php will parse it, everything from player name to unlimited. I think one of scorpion182's map uses it..

Anyways

Cooler than before >

 

[AnonymousPro]

How do I use it? Can you teach me I'm a beginner...

 

[Nestharus]

This map shows all ;o

http://www.hiveworkshop.com/forums/...221199eff1f14b315571b1ebf&dateline=1354588992