SRL Archive - General

The mysterious invisible dot


riwu

2012-06-17 00:14:27

Take a look at http://paste.villavu.com/show/2991/
Just 22 lines of blank space right?
Now select(highlight) the blank spaces and copy and paste to ur simba!

Amazing right? I have absolutely no idea what these mysterious invisible dots are, they originated from my script on googlecode.
I uploaded the script to googlecode, then when go edit then copy and paste the script to simba, and to my surprise it didn't compile and highlight the line with that small dot (which appears out of nowhere). But using autoupdater (GetPage function), that small dot will not be retrieved and the script will compile fine.

Any expert know what this little dot is?

Recursive

2012-06-17 00:27:02

Weird…what does it mean? AHHH!

Gucci

2012-06-17 00:40:58

Wow that's pretty cool, fact that it actually creates a compiling error is weird

Brandon

2012-06-17 00:48:03

Iunno what you guys are talking about because when i copy it, it copies nothing.. just a bunch of blank spaces. Pasted it into simba, still just 22 blank lines. No dots, no characters, no anything.

I copied both raw and normal view of the file. Still nothing. Using firefox 13.

riwu

2012-06-17 00:52:54

From: Brandon
Iunno what you guys are talking about because when i copy it, it copies nothing.. just a bunch of blank spaces. Pasted it into simba, still just 22 blank lines. No dots, no characters, no anything.

I copied both raw and normal view of the file. Still nothing. Using firefox 13.

hmm try use chrome?
I'll put the code in simba tag & upload a copy of those dots, see if u can see it.
                                                        
                                     
                    
                                            
                                     
                      
                                
 
                    
                                         
               
                                         
                                                        
                                                        
                                                        
                                                        
                                                        
                                                        
                                                        
                                                        
                           

EDIT: wow putting the dots in simba tag returns ^!!

John

2012-06-17 04:09:04

For those that it doesn't happen to.

Sirenia

2012-06-17 09:05:40

Feels like we discovered some mystery within Simba :O

Brandon

2012-06-17 09:23:29

This looks like the difference in format.

Unicode vs. Ansi. I'm pretty sure Simba's text editor is by default UTF8.

In Simba a Unicode string is:

var
US: UnicodeString;

vs.
var
AS: AnsiString;

vs
var
Str: String;

The difference is that the unicode string supports more characters. String should be the #Define that assumes both types via the OS. If your OS supports Unicode then string should be unicode else Ansi.

Now of course this is just a theory but meh. It CAN screw a lot of programmers. WideChars vs. Chars, etc. The browser encoding probably differs from that of Simba's editor. I still cannot replicate it on my machine though.


Edit:
I replicated it lol. It is a unicode difference. That character code is the UTF-16 Zero Width No-Break Space; HTML Decimal form. So it's a huge difference that can definitely ruin a programmer. Simba should support Unicode-Wide Chars if it doesn't already.

UTF-8 isn't enough.. UTF-16 is ok. UTF-32 is Standard.

riwu

2012-06-17 22:18:36

From: Brandon
This looks like the difference in format.

Unicode vs. Ansi. I'm pretty sure Simba's text editor is by default UTF8.

In Simba a Unicode string is:

var
US: UnicodeString;

vs.
var
AS: AnsiString;

vs
var
Str: String;

The difference is that the unicode string supports more characters. String should be the #Define that assumes both types via the OS. If your OS supports Unicode then string should be unicode else Ansi.

Now of course this is just a theory but meh. It CAN screw a lot of programmers. WideChars vs. Chars, etc. The browser encoding probably differs from that of Simba's editor. I still cannot replicate it on my machine though.


Edit:
I replicated it lol. It is a unicode difference. That character code is the UTF-16 Zero Width No-Break Space; HTML Decimal form. So it's a huge difference that can definitely ruin a programmer. Simba should support Unicode-Wide Chars if it doesn't already.

UTF-8 isn't enough.. UTF-16 is ok. UTF-32 is Standard.

But how did an Unicode character end up in my script? All i did was:
1. Upload a (successfully compiled) script to googlecode

2. Wanted to double check nothing went wrong with the upload so i went to googlecode and use the edit so that i can just select all and copy paste the script to simba.

3. I tried to compile it which was how i discovered the dot (which caused compiling error)

4. I went back to googlecode and checked the line, but there is no dot on the line (apparently the dot is seen as just a blank space in web browsers), but when i copied and pasted the whole line to simba the dot appeared.

5. But if i were to copy and paste the line on googlecode without using the EDIT feature there will not be any dot!

Is it an intentionally/accidentally imposed dot on the EDIT version due to some coding errors/limitations on googlecode's part? That dot is recognized as blank space on notepad/microsoft word too, only simba managed to detect it as unicode.

Brandon

2012-06-20 14:38:33

From: riwu
But how did an Unicode character end up in my script? All i did was:
1. Upload a (successfully compiled) script to googlecode

2. Wanted to double check nothing went wrong with the upload so i went to googlecode and use the edit so that i can just select all and copy paste the script to simba.

3. I tried to compile it which was how i discovered the dot (which caused compiling error)

4. I went back to googlecode and checked the line, but there is no dot on the line (apparently the dot is seen as just a blank space in web browsers), but when i copied and pasted the whole line to simba the dot appeared.

5. But if i were to copy and paste the line on googlecode without using the EDIT feature there will not be any dot!

Is it an intentionally/accidentally imposed dot on the EDIT version due to some coding errors/limitations on googlecode's part? That dot is recognized as blank space on notepad/microsoft word too, only simba managed to detect it as unicode.



The reason why other editors leaves it as a blank space is because it is a blank space in UTF-32 Editors. Anything else is going to show it as a weird character. Github sometimes does this too (For code edited in DevC++) Whereas if edited in codeblocks, it will replace it with a space or a tab.

It's the minor differences that screw us over. Think of it like this: I draw out the blue prints for a rocket and I measured everything in Inches because I like the British Standard. Mind you I did not write the standard measurement beside it. Instead I wrote 32 with no Measurement syntax at the end.

You on the other hand got these same blue prints through the mail but since you live in Iunno, the US, you assume it to be Metric instead of Imperial. You build the rocket and everything seems fine, you launch and everyone is dead and you wanna know why.

Well it's because we both had different standards and it's my fault for not putting the Standard declaration at the end; such as: 32" would signify 32 inches/imperial whereas 32mm or cm would signify metric standard.

Unicode works the same way except it's usually signified by a range of characters such as 1 - 256 vs. 1 - 110k. Sometimes even UTF-8/16/32 will signify it's support. Code editors have the job of deciphering what format your code is and you as a programmer should support portable code by using a standard format.

ANSI is old so it can be used on very old machines whereas Unicode can be used on anything Win98+ but at the same time ANSI supports less characters than Unicode of a specific Bit type.

BTW, my little rocket example happened in real life. ~ NASA ~ -__-. Also screwed me over once! when I did Mech Engineering.

TLDR:
Simba needs to update it's code editor OR change all string types to wchars and wstr/wstrings.

Wizzup?

2012-06-20 15:10:21

From: riwu
But how did an Unicode character end up in my script? All i did was:
1. Upload a (successfully compiled) script to googlecode

2. Wanted to double check nothing went wrong with the upload so i went to googlecode and use the edit so that i can just select all and copy paste the script to simba.

3. I tried to compile it which was how i discovered the dot (which caused compiling error)

4. I went back to googlecode and checked the line, but there is no dot on the line (apparently the dot is seen as just a blank space in web browsers), but when i copied and pasted the whole line to simba the dot appeared.

5. But if i were to copy and paste the line on googlecode without using the EDIT feature there will not be any dot!

Is it an intentionally/accidentally imposed dot on the EDIT version due to some coding errors/limitations on googlecode's part? That dot is recognized as blank space on notepad/microsoft word too, only simba managed to detect it as unicode.


Paste it from raw view, not the html view.
I don't know any serious language that uses unicode as delimiters.

This is not a bug. Try to encapsulate it in a String. You'll find it works just fine.

Wizzup?

2012-06-20 15:18:18

From: Brandon
Edit:
I replicated it lol. It is a unicode difference. That character code is the UTF-16 Zero Width No-Break Space; HTML Decimal form. So it's a huge difference that can definitely ruin a programmer. Simba should support Unicode-Wide Chars if it doesn't already.

UTF-8 isn't enough.. UTF-16 is ok. UTF-32 is Standard.


This is simply not true. I don't know anyone or anything that uses UTF-32. Correct me if I'm wrong. Simba supports Unicode just fine, but not outside anything other than a string, obviously. Try doing it in C. Or Python. You don't use unicode in code, just in strings.

I don't think there's anything wrong with the Simba code editor either. If you can't see the character; you lack the correct characters in your font.

Example:


program new;
begin
writeln('I ♥ ♫');
end.

Results in:


Compiled successfully in 409 ms.
I ♥ ♫
Successfully executed.

E: But perhaps I misunderstand… :)

Brandon

2012-06-20 15:33:15

http://i.imgur.com/LeVcn.png

https://github.com/MerlijnWajer/Simba/issues/120

Simba isn't identifying the characters as one in the same. I explained better on the opened issue.