Base64 encoding and its proper use

A friend of mine recently unveiled a new version of his website. Given that I too had worked on it sometime back, I decided to check it out. The main points I looked out for was security holes, since there were numerous such flaws which I fixed years ago.  Rather unsurprisingly the password reset feature of the site had one such ‘vulnerability’. It was done anew and the method used was not the most recommended although many developers opt for it. It wasn’t a bug, but a bad implementation ready to be exploited.

The culprit was an incorrect use of base64 encoding. For me, the main uses of base64 are storage and transmission of non-secret data. Although in the case of storage it’s something like hashing binary data etc. The last part ‘non-secret’ is very important. Because if one was to use the same for ‘secret’ information, then again it’s a non-recommended use. In the above scenario that was the exact thing that happened. A piece of data which was meant to be secret and easily non-readable was sent publicly after encoding with base64. And as most developers know, it’s just a jiffy to decode base64. So what I did was decode the string value, only to find out that two values were concatenated. One value was the victim’s email and the other a randomly generated string which wasn’t that hard to identify. Then it was merely to modify the above value with a known user’s email. And voila! I could reset his/her password.

Mentioned above was how a badly implemented encoding could make your web application vulnerable. And this is not something associated with low-profile companies, but even Facebook had a similar situation which was revealed in this article at 'Hacker News'.

The remedies are many, depending on how far you’d be content with given that security isn’t a 100% achievable thing. One solution is to make the random ‘salt’ a highly cryptic value. Another is to use a well-recognized encryption mechanism. Or you could even develop your own encrypt function although security experts warn against this. A rather straightforward and often used method is to implement one-way hashing such as MD5. All this methods have their advantages and perils. It’s up to the developer to decide which is best depending on factors such as performance, importance, accessibility, etc.

A week of tech meet-ups

It's not often that you find yourself attending several tech-related meet-ups during a week, especially in Sri Lanka (let alone happening). But, last week turned out to be exactly that. If there is a trend building as such, then that's great and I hope for more. These provide an opportunity to share, learn and network with people offline as oppose to Googling or Social Networking. It also reminds some that they are humans!

First was at Coco Veranda on Monday 23rd January. This was arranged through Meetup.com by a bunch of guys from a company called WebGurus and was meant for PHP developers only. Being the first of its kind I attended, (and probably the first in Colombo) there were about 15 - 20 PHP developers gathered that evening. The session commenced with each person introducing him/herself while the main feature of the evening was a presentation by Jayawi Perera on the security aspects of PHP development, which was pretty good. Afterwards, it was more chatter within the gathering to go along with Coco's delicacies (there were many). It was agreed to hold a similar meet-up every month to discuss PHP related stuff.
 

Come Thursday, and it was time for Refresh Colombo. This was the first 'Refresh' for 2012, and two talks and an inaugural networking session were lined-up for the evening. Needless to say, there were about or probably more than 100 people attending this time around. Personally I was glad to meet an old friend Dilshan Kathriarachchi after about seven years. One of the talks scheduled for the evening was by him. The first presentation was an SEO audit of a local website by Shaad Hamid. He discussed a few SEO points and shared some tips in between too - which SEOs normally wouldn’t do. Then it was Dilshan. He talked on his experiences with start-ups and the journey so far. I had stuff to reminisce during the talk too, because it was he and his first start-up that I first worked with eight years ago. Those were crazy and fun times. Lastly it was a networking session, where we got to know others that we hadn't met or talked before and discussed what they were up to.

  



From two-digit attendance to three-digit attendance, and the last for the week was easily a four-digit one.  The second 'Android Forum', organised by Etisalat Sri Lanka, it was. This was held at the Cinnamon Lakeside hotel on the 27th of January. The morning programme was meant for Android developers, whereas the evening session was aimed at explaining the ABCs of Android. There were some fine presentations in-between and a useful panel discussion as well. The guy from Google Israel was something special too.

All-in-all it was a week full of knowledge to harness, people to connect and stuff to share. One thing unique about all these functions was that they were free of charge. So it is only proper to sincerely thank all the wonderful people behind the sponsoring & organising of these events.

Image sources: -
Meetup of PHP Developers, Sri Lanka Facebook page
Refresh Colombo Facebook page
Android Forums Sri Lanka Facebook group

My AI (Artificial Individuals) Connections

10 years ago I got my first PC. I was still schooling. As most teenagers would do, I spent most of my time playing computer games. However after awhile, I was very good at discovering cheat codes which resulted in almost every game finishing very quickly. This was boring.

Then one day, I received a little program named “Billy” which ran in DOS mode. It would easily fit into a floppy diskette and is one of those quickly executable types. What was it? An artificially intelligent (AI) computer program that I could chat with (in text mode). Billy was an exciting experience for me. I would type-in something to him, and he would respond, not intelligent always. The guy sounded dumb after a few chats because his response was more or less the same for a similar question or phrase by me. But, to my delight I was able to discover and download Billy’s female version – “Daisy”.

Daisy was much intelligent than Billy, this is because the program and its “brain files” had evolved by then. So, it became a habit of me to chat with these two AI bots daily. In the process, they learnt from me more than I did from them. While all this was going on, the creator of Billy & Daisy had mentioned a method of connecting 2 or more bots together so that they could communicate between them autonomously. Therefore, I decided of linking Billy & Daisy to see what happens.

All I had to do is connect the two programs and watch them chat. It was fascinating. They were talking on their own and every now and then I would see knowledge snippets learnt from me being thrown at each other. Guess what? I even got to know that Daisy was in love with me!

Sometime later my hard disk crashed and all my software/data were gone, including these two programs. They were no longer available to be downloaded, hence I lost them. (Please email me if buy any chance you have it). So that marked an end of an escapade.

Two years later, I got my first job. As a web developer I was more often than not spending time on the internet. This is when I came across a Sri Lankan chat room. Soon I was a regular member of this community and the owner of the site (who was now a good friend of mine) was kind enough to present me a moderatorship of the chat. He also offered me the opportunity to play around & develop modifications or additions to the site.

By this time my PHP skills had greatly improved together with the knowledge of this Open Source chat system which was pretty popular at that time. Unsurprisingly, I was experimenting of a way of creating an artificial user who would respond to others in the chat room. And so this is how “Ecila” was created. Ecila if pronounced reversely is Alice, and that wasn’t unintentional thanks to some amateur naming skills ;-). After all, ‘Alice’ seems to be the most popular name used for chatter bots. Read till the end and you’ll see why.

This was early 2004 as I remember. Ecila would occasionally log into the chat even as Alice, thanks to a “cronjob” setup by me. Thereafter she would have a brief stint (about ½ hour) with the guys in the room and exit gracefully. However, regular visitors to the chat found her to be weird; a person who answers similarly more often. Why? That’s because she was only about 150 lines of code in length recognising only a limited number of phrase patterns together with their related answers.

Then around June that year, the chat was facing moderation problems due to an increase of swearing. We needed an ‘automated’ moderator who’d keep an eye throughout the day (24 hours) by detecting these mischievous users and consequently warning or banning them. So I decided of revamping Ecila by augmenting her already available functionality and adding a few other dimensions. These included the ability of being active all the time, dealing on multiple users, tracking user history and decision making based on factors such as history, gravity of harm caused and spontaneity of misbehaviour. After all was ok, this new bot was named “No Hellụm” (meaning immovable in Sinhala) and was logged into the chat as a moderator. It was a totally new concept and was a very productive idea. Although after quite sometime users figured out that he (in fact No Hellụm has no gender) was non-human, the occasional loner would always converse with him as well. This version was revised again in 2006 and now No Hellụm acts as the chat-keeper and has the ability of providing real-time score updates of cricket matches too!

It was the latter part of 2005 when my next endeavour commenced. A chatter bot named “Alice” was plugged into my friend’s website’s homepage. However we had no access to its source code as the scripts were hosted elsewhere while we just provided the interface through an ‘iframe’. This Alice was pretty good and intelligent. So I began searching for her and discovered that it was an open source project downloadable at ‘SourceForge’.

The bot uses a set of AIML files to initially populate its database with ‘knowledge’ (patterns & templates); more the patterns, more intelligent it becomes. Hence, I began searching for more AIML files too. AIML is an XML based standard which maps linguistic patterns to how they need to be dealt with, and in no time I was modifying Alice’s personality and writing a few of my own patterns & templates in AIML which I eventually fed to her. Currently the version I have is enriched with almost 45,000 templates. Furthermore, I developed a couple of new tags for my needs. One which added the capability of outputting emoticons in the response and the other enabled simple administrative functions.

Enable administrative functions? Gave it a thought? This is the first of its kind and most probably the first time you are hearing of it. What I did was, by modifying the code I was able to use her as a virtual login interface which would permit me to execute simple admin (database) queries on her and virtually logout when I’m finished. For example, I would chat to her as follows:

Me: login me
Alice: What’s your username?
Me: Udendra
Alice: Udendra, what’s the password?
Me: ********

Now I’m virtually logged in, and Alice knows it for sure, cos if I’m not she wouldn’t allow me to perform the following queries:

Me: show users (who are connected)
Alice: <<displays a neat user list>>
Me: show previous chat 10
Alice: <<displays the last 10 records in history>>
…………
…………

When I’m done, I could logout:

Me: logout
Alice: You have been logged out. :-)

This in fact pushes the limits of interaction between a computer program and a user. The beauty of it is that one could chat to the bot in layman’s language in order to login to it, perform various functions, and logout.

I guess by now its clear why my title reads as ‘Artificial Individuals’ rather than ‘Artificially Intelligent’ that most people would expect. Simply, this is because none of these programs were/are fully artificially intelligent. To become a truly AI entity is a great achievement that illustrates signs of intelligent behaviour which are thinking, reasoning, envisioning/perceiving, human performance, rationality, learning and adapting. However, what’s important is that the aforementioned programs had varying intensity of AI features such as searching, knowledge representation and learning and at times were extremely impressive in several areas.

So the next time you feel lonely and there’s no one to chat with, don’t give up hopes. Who knows, an ‘artificial individual’ may fulfil your need.

You have a new text message (email)!

Few months ago I was having a requirement of getting notified whenever my office mail box received an email. The preferred notification-media was SMS. So basically whenever there was a new email to my official email account, I would get notified with its subject and sender via sms.

Now then, there were several obstacles I had to conquer in order to accomplish this task.

1. Accessing & Reading my email inbox hosted at the place I work.
2. Accessing it frequently (at least every few minutes).
3. Hosting the program (script) that would do the above, on a server that’s available throughout the day.
4. Protecting sensitive/personal data from third parties (in this case it was my password).

Our office uses a Microsoft Exchange Server as its mail server. Performing programmatic tasks on MS Exchange using PHP was a very rare occurrence. So I had to rely on Mr. Google in seeking a suitable code snippet that would help me in this endeavour. And guess what, I did stumble upon a blog on how to use XML-formatted WebDAV requests to fulfil my requirement.

Building on it I developed a modified version of that script, which would send an XML request to the server, read the response (it was XML too), process it, check for new mail messages, log it (because I need to keep track of what’s new & old), and sms it to my mobile. An sms was sent for each email in case of many.

I could see the inquisitive mind wondering… ;-)

A facility called ‘email2sms’ provided by my network operator was utilised in delivering the sms to my handset. This whole process was carried out every 5 minutes and the scripts that were responsible of performing this function were hosted on two web servers hosted elsewhere. That’s because my office PC is powered on during office hours only and email relaying to external destinations aren’t allowed. Moreover I don’t have permission of using the office web server for this purpose. Keep in mind that these weren’t mentioned as obstacles at the beginning of this article, as this doesn’t serve any official purpose but one of my ‘private’ needs.

Hence, I had to host the scripts on two web servers situated in different places, they both belonging to a couple of my friends (thanks guys). These servers are operational 24*7 and the every-five-minute request was handled by a simple Linux cron job.

Did you notice me mentioning 2 web servers? Wondered why? This is to overcome obstacle no. 4. In the aforesaid script there arose a need of hard-coding my email account’s password into it. This is required for the script to access my inbox since credentials need to be given. But if anyone was to open up and read the script (this wouldn’t normally happen, but my contentment of information security wouldn’t be satisfied otherwise) my password was there for his/her taking. To avoid this happening, I considered the use of cryptography.

Using the MCRYPT_CAST_256 cipher I was able to generate the encoded (cipher) text and a relevant key. The key was hosted on the server that initiates the request while the decrypt function and cipher text were on the second. The latter would perform the core activities mentioned earlier. Splitting these elements was necessary as it would make no sense to have the key, cipher text and decryption algorithm in the same place. If anyone was to try fetching the password on the second server by performing a ‘cipher text only’ attack, still they would need an extensive amount of brute-forcing to be performed. However this was evaluated by me as computationally secure for a long time.

Something worth mentioning is the disguising of the key when it’s transmitted from the first server to the second; care was taken to avoid any logging on the latter’s part so that no readable trace of it was leftover.

Although I’m not using this service these days (cos spammers occupy a decent share of my inbox ultimately filling up my sms inbox too!), the learning and application of learned theory was something to admire in solitude.