Friday, October 31, 2008

"Hotmail" the fastest of them all

I have been using Hotmail, GMail and YahooMail for a very long time now. And boy was I surprised by the speed of the newest version of Hotmail. I have these three pages setup as part of my firefox home page. And for a very long time the the order in which everything would load up was:

1) YahooMail: Yahoo has constantly been fast in serving up the pages.

2) GMail: The second fastest to open was GMail. But GMail more than made up for the startup time by making all the other operations fast. And then you have the gtalk embedded into GMail which too is very fast.

3) Hotmail: The last to open in was always Hotmail. It displayed a page where it would ask if you wanted to open it's classic version.

But now just a couple of days ago, I logged into Hotmail and was completely shocked (pleasantly) by it's speed. And it has also got some stunning themes. This is the best version of Hotmail I have ever used. The speed simply knocks your socks off. Looks like Microsoft has finally got its act together. If you haven't tried the new Hotmail, I suggest you to take a look at it, You will be pleasantly surprised by this new thingy.

\10-31-2008 5-27-09 PM

Sunday, October 26, 2008

Revamped Delimited File Viewer

My life has been going through a lot of turbulence lately. I have left the company for which I used to work (Tata Consultancy Services). Now I am working on starting a new company. But, I am unable to get a hold of things, So, I have planned to finish all the things pending with me :D

First I am planning to revamp The Delimited File Viewer completely. I have got a lot of requests from you guys about the changes to be made to this utility. I will be finishing this in a couple of days (Hopefully :)). Let me know if you guys have any inputs.

Thursday, July 24, 2008

Sneak peek at dsSurfer!

Well, things are not turning up at the time I would want them to. Anyway, I was able to finish some of the stuff and just wanted to share it with you guys. Here are a couple of screenshots of dsSurfer:

Will upload the first version very soon :D

Wednesday, June 18, 2008

Schedule for the open source Datastage tool (codename: dsSurfer)

One of the things I have always wished to do was, to work on an Open source project and get to interact with the peers in my domain. Thanks to DSXchange my wish has been partially fulfilled as I get to interact with the best minds working on Datastage. There are quite a lot of reasons why we don't have open source projects in Datastage, I am not going to dive into those details.

The motivation behind this initiative is my wish to make open source projects in Datastage a possiblity. Once I decided that I am going to start an open source initiative, All I had to do was dive into the posts at DSXchange and find out the most common things that Datastage users wanted. My first open-source project is going to be the creation of a few utilities which would help the developers do a few things using "an easy to use GUI". With that said, let's jump into the windows utility which we are going to create. I have given it a codename dsSurfer (it isn't very cool, But I am expecting you guys to help me out with the actual name). Now dsSurfer would be a Windows application (similar to DelimitedFileViewer) which would have a GUI just like any other desktop application and it would work from desktops which have Datastage client components installed on them. And it would broadly do the following mentioned below this para. Please post comments on what *else* you would like to see in this tool and we'll see if we can accomodate them into this release.

  • Launch Datastage client components.
  • Generate job reports
  • Schedule backups of projects

Also, I would need a lot of help from the experts at DSXchange in figuring out the best possible way to do these things. I have also planned a schedule for the release dsSurfer.

# Schedule

  • 06/18: Post it on the blog
  • 06/20: Freeze the requirements and post the final requirements on my blog.
  • 06/25: Finish the UI
  • 07/05: Finish the coding for req 1 and req 3
  • 07/15: Finish the coding for req2
  • 07/17: Finish the complete integration and test it on 7.5x2 and 8.x thoroughly.
  • 07/18 - 07/20: Alpha testing
  • 07/20 - 07/25: Beta testing
  • 07/26 - 07/28 : Bug fixing
  • 07 /28: Final Release

This tool would have the abilities to:

Generate job reports

  • It has to have the ability to search jobs and generate reports of jobs selected from the list box.
  • If one of the jobs is a sequence, it needs to generate a report listing all the jobs used in the sequence.
  • A way to integrate all the reports into one html document?

Schedule backups of projects
Options to schedule backups of:

  • All projects on a server
  • Selected projects from a server
  • Optionally zipping the dsx files

Launch Datastage client components

  • Store the user id and password associated with a datastage server (Use encryption to store the passwords).
  • Launch Designer, Manager, Administrator and the Director.
  • Options to store as many server/user combinations as possible.

Technology to be used:

I am going to use C# 2.0 to write all the front end code and may be use the Datastage API to get the information needed from Datastage. Also, I'll be putting up the screenshots, the pieces of code which I come up with, on this blog. And hopefully by 28th July 2008, we will have a cool tool (with a cooler name ;) ) to make life in Datastage easier and more fun.

Please post your valuable comments on the requirements and other stuff.


Updates on 06/23/2008:

I was expecting a lot of suggestions on dsSurfer :O (I got a couple of suggestions from Amey and Uma Maheshwar) But, I guess I was too overzealous about this whole thing. Since the requirements have not changed, I thought that just updating this post would do.

I'll go ahead with the requirements posted above, Very soon (hopefully ;) ) you guys will get a peek at what's in store for dsSurfer. I'll be posting the screenshots of UI once I am done with it. Hope to recieve your continuos support ;)


Tuesday, May 27, 2008

End of the "Road to Datastage certification" :D

I feel very happy at the end of my journey :D (So, unfortunate that there are no emoticons in blogger ;) ). I finally("finally", because I have been planning to take this for a long time now) cleared my Datastage Certification. There are a few things which give you the joy comparable to getting your certification. I want to thank my friends Ajay Prakash and SubhaKarthik for giving me the moral support for this certification.

I had blogged (Road to Datastage Certification) about how all this certification stuff started. I had planned to take the certification on 26th of this month. I applied for a leave on this day, to buy myself time to read the documentation for my certification. Booked the test at 5:30 PM in the evening by taking a cue from Vincent's experience with the exam. I started the day well by getting up at 6:30 AM and jogging for some distance. I was preparing happily when this guy from the prometric center calls me up at around 2:00 PM and says that I won't be able to take my certification because of some problem with their server (From when did Prometric centers start having servers of their own :?, I thought it might have been a problem with their internet connection). He asked me to call him up before I started to the exam center, in case the server issue gets fixed. I was very disappointed on hearing this, as I was highly pumped up to take the exam. Anyway, I kept reading the documentation, and trying out things on my VM. At around 5:10 PM I called up that guy, and got a happy reply from him saying that the server was up. I quickly went through the important notes I had jotted down during preparation (No matter how much preparation you do, you will never be "satisfied" with your preparation ;) ). I got on my bicycle in my tracks and a T-shirt and got off to the certification center. I reached the center by 5:25 PM (It's about 250 yards from my home) and, by the time I got in front of the computer to take the test, it was 5:38 PM. I finished the exam in about an hour and reviewed all the answers in the next 15 minutes(finished 30 minutes before the end time). I clicked on the End button with my adrenaline on an all time high!! I was on Cloud number "99" when I saw that I passed the exam with a good score. I thank God for this, and happily come out of the exam

As for the exam, It was a moderately tough exam, It had 79 questions most of which had multiple answers. There were also a few questions which ask you to drag and drop the right boxes in the options given (I was impressed ;) ). In all, if you prepare well and have some good experience in developing Parallel jobs, debugging jobs and administering Datastage, the exam shouldn't be very difficult. If you don't do all that, don't panic, go through the documentation which comes with the Datastage client, that should help you out with things you don't do.

This was the split up of questions in the exam:

Section/Category Number of Items
  1. Installation and Configuration 6 out of 79
  2. Metadata 4 out of 79
  3. Persistent storage 10 out of 79
  4. ParallelArchitecture 10 out of 79
  5. Databases 9 out of 79
  6. Data Transformation 11 out of 79
  7. Combining and Sorting Data 8 out of 79
  8. Automation and Production Deployment 7 out of 79
  9. Monitor and Troubleshoot 7 out of 79
  10. Job Design 7 out of 79
  1. Go through the installation and configuration guide, Especially the configuration on a Unix machine and the configuration of Enterprise Edition on USS. There were a few questions about the Unix Kernel settings, so don't ignore them.
  2. Metadata would be an easy section, Read about different types of datatypes in Datastage, ways to import metadata. Also know about different OSH datatypes.
  3. Persistent Storage is a section which asks questions about Datasets and Filesets. This should be another easy section if you have played around with Datasets. Know how data is stored in these two files
  4. Parallel Architecture is a huge thing and you should read the Parallel job developer's guide and the Parallel job Advanced developer's guide to score in this section. There was a question about conductor nodes, section leaders and players, which was taken straight out of the Parallel job developer's guide.
  5. This is one section which consumed a lot of my time for preparation, I have just used Oracle as the database in my Datastage jobs. But this is not enough, you are supposed to know "everything" about Oracle, DB2 and Teradata (These were the only databases on which I got questions). Read through the complete documentation for these stages, and don't forget the environment variables which effect these stages.
  6. Data Transformation is an easy stage if you have created even a few jobs in Datastage Parallel edition, This should be the section with which you are most familiar. Questions in this section can be answered with a little preparation.
  7. This section has a lot of questions about the sort stage, join, merge, lookup and aggregator stages. Make sure that you know all the environment variables which effect the performance of these stages. Also go through the differences between similar stages (such as the differences between Merge and Lookup).
  8. This is an interesting section which covers Job Sequences (which are not documented in the Parallel Job Developer's guide!!) and stuff about the command line interface to Datastage. Know all the things possible with the use of "dsjob" and similar commands.
  9. This section had a lot of show and tell questions. It had a question which had an exhibit of a job log and asked us to pick up the log which has the OSH code. For this section go through all the Environment variables present in the Reporting category and know how each variable effects the job log.
  10. The last section (Am I happy :D) , It has questions in which you are asked to optimize the design of a job.

On a scale of 1-10 for toughness, I would say that this test stands at 8.5 (I love fractions ;) ). It is tough in that it tests the breadth of your knowledge in Datastage.

I was a guy who didn't belive in certifications, I was of the idea that no certification can measure your actual skill on a Tool, because with enough time and with "NO" practical experience, you can get a pretty decent score in any certification. I guess I was wrong, These are a few benefits I didn't know about a Certification:
  • You get a LOT OF CONFIDENCE when you are certified, it gives you a benchmark against which you can compare yourself with the rest of the guys.
  • More importantly, it fills a lot of gaps in your knowledge. I can't really list out the technical stuff I learnt during the preparation for my certification. This makes you a better developer however good you currently are.
  • It is a humbling experience (It was for me!). You get to see the things which you don't know, It's like a reality check which is good to have. And it opens up possibilities for your jobs, You connect to the inquisitive child within you during the preparation.
  • It gives you a very good break from the routine stuff which you do every day.
  • Finally, it may get you a better job :D
If these things don't motivate you enough, then you got some serious problems ;)
I would encourage people to take certifications (doesn't matter which certification you take). Now, I will stop my lecture about certifications and put down a few useful resources which helped me with my certification.

Resources I used:
Please leave comments if you find this post helpful, or if this post "helped" you in wasting time ;)
That is all I have for now, Whew! Blogging is no easy thing!

Saturday, May 3, 2008

Road to Datastage certification - Day 01

I have been planning my datastage certification for quite some time now.
I am planning to take it up on 21st of this Month (05/21/08). So, I have around 17 more days. And, I will be posting about the topics and random stuff (which I hope would help somebody some day!) which I use to prepare for the certification . Here is a link to a Google public notebook, which I thought of using to document my journey ( But, I scrapped that idea in favor of blogging) Road to datastage certification, This notebook has links to some very good resources.

Before I begin, here is some background info on how it all started. On 05th of March, My friend challenged me that I could not complete any certification (I had paid for an Oracle PL/SQL my vouchers had expired, Was very lazy to take up that certification). I said that, it was a piece of cake for me ( I bluffed ; ) ). We thought two months was a fair amount of time to prepare for certification. The bet was worth Rs 500 ($12.50), And I would win it if I scored more than 80% in the certification, and my friend would win otherwise.

Two months passed in no time. On 03 of May, I told my friend that I wouldn't be able to finish my certification and would need some more time to finish it. My friend, being the generous guy he is ; ) changed the date to 27th of May. But, there was a catch, I had to pay him Rs 250 now, and pay another Rs 750 on 27 May, if I lost, If I won, I would get back my Rs 250. I readily agreed, and here I am on my "Road to Datastage Certification"!!

I have a rough plan in my mind. I would want to finish all the stages in two days, i.e. 05/5 and 05/06. And, then I would plan to tackle the topics in the certification. At the end of my certification, I am planning to put up all the demo jobs and the knowledge I gain from this certification on my blog.

Tuesday, April 29, 2008

Delimited file viewer (A utility to read delimited files)

I have been using this little utility quite for some time. And I thought It was about time I shared it with the rest of my community.

It's called DelimitedFileViewer (Was unable to come up with a funkier name;) ). I had created the first working version of this way back in November 2007. After a lot of cosmetic improvements, it works well and looks good. There are a lots of features like Finding stuff and Sorting stuff on one of the fields. It also has a nice drag and drop feature which lets you drag and drop files onto the application. Click on the following link to download this utility
The following are a few screenshots:

Without any file:

With a test file:
After sorting the test file on the emp field