Monday, December 18, 2006

Are you going to be graduating soon?

"If so, how many jobs are you going to create"

This is a poster in a popular US university, which in a way is encouraging youngsters to be entrepeuners and risk takers. This was mentioned by Dr. Mashelkar (President of CSIR labs in India) in his talk on how India can rise to the challenge of creating an environment to nurture ideas to become economic succcesses. This was at the inauguration of the NCL innovation part in Pune, where they plan to nucleate and nurture technology and knowledge based enterprises and support research projects which have a potential of growth in the global markets.

He mentioned how the youngsters in India are beginning to think differently and are willing to take risks to follow through their dreams. Of course this calls for a major culture change in the mind set of the youngsters.

Prof. Sir Richard Friend from Cambridge, who has launched two successful ventures from the small town of cambridge mentioned now a town with a population of 100,000 has 900 small companies generating an employment of 27,000 (averaging 30 people per company). Each of these small companies is a valuable asset to the UK nation, since they have a solid potential to scale economically thus enabling wealth creation for the people and the nation. Of course its a daunting task to convert science to technology to products which can be mass manufactured, but all the speakers stressed on the importance of enjoying your work through the pains and the joys of being cashless, struggling to acquire and retain talent, fighting off the predators (read large companies with full koffers), innovating to maintain leadership, and maintaining gumption through all this for a rewarding journey which in itself is the biggest takeaway.

Dr Mashelkar mentioned how Chinese universities have successfully incubated 2800 small businesses generating employment for about 90,000 people (again averaging around 30 people per company).

The question to be answered is, that if USA can do it via Silicon valley, Europe can do it via a small town like cambrdige, Asia can do it as demonstrated by China, so why cant India ? I think we the people have to do some introspection into our basic values and create a culture of loyalty and commitment, risk taking, wealth creation (not the same as earning via jobs) to lead a more fullfilling and visible life.

Thursday, June 08, 2006

Amazing Himalayas | Amazing Holiday

Hi Friends
I'm just back from a very refreshing trip deep into the Himalayas. This was a 15 day outing with 7 days of heavy trekking thrown in. Trekking through the mountain ranges in the North western part of Uttaranchal (Garhwal region) is an experience where you discover yourself. You are in totally disconnected territory with no electricity, no connectivity, no motors - just basic living where you walk, eat, sleep. The note below describes our journey briefly (I went with a friend Sunil Khanna, who shares my passion for the outdoors) and we had a ball.

1. Reaching base camp:
This was a journey through the city jungles via planes, trains and buses. We went for this trip with a group called Yuvashakti, who are Pune based and organize treks and outings. We were a group of 36 participants (which I thought were too many), with a mixture of young, middle aged and old people. Most of the participants chose to come by the train from Pune to Delhi (we chose to fly to save a business day). After an overnight halt at Delhi, we proceeded to Dehradun by train. Temperatures touching 45 deg Celsius caused us to sweat buckets. In the 6 hour train journey, I drank 3 liters of water. In Dehradun, while most of the participants stayed in the hotel accommodation arranged by the organizers, we chose to stay with our friends in their beautiful picturesque home overlooking the valley. Next day morning we set out in a bus to Sankri covering a distance of about 280 kms mostly traveling along the mountain ranges of Uttarkashi district and along the Yamuna river. This was a back breaking journey through broken roads, gravelly roads and sometimes no road (due to frequent land slides) (This part of the Himalayas is landslide country). This 12 hour journey gave us an opportunity to get to know each other and amidst a lot of singing and chatting we proceeded towards Sankri, the last motorable point along the route.

At Sankri we stopped at the GMVN (Garhwal Mandal Vikas Nigam) (Government managed guest houses with basic amenities like beds, toilets and a canteen). We were greeted with news that our booked dormitories were allotted to some other guests and there was no solution coming forth from the staff in charge. The participants patiently waited while the group leader tried to sort out the issue. After about an hour we saw a stalemate situation and then some of us tried to intervene and after 3 hours finally managed to get a dormitory for the ladies and quarters of the GMVN employees for the men. Was uncomfortable to say the least. Sankri was cold and on the instructions of our group leader, tried to acclimatize by delaying wearing our woolens.

A note about the organizers: Previously Yuvashakti would book these GMVN guest houses for a fixed period and have a resident camp leader who would oversee the arrangements and handle the traffic of the batches of participants. However, this year onwards they decided to outsourcing the entire arrangements to a local group of people run by a leader named Nautiyal. I guess they paid him per participant and expected him to handle all logistics of staying, guidance and food from base camp to base camp. This incident however showed how such an arrangement can go sour in the absence of local supervision and having reached so far there is no way but to go along. These local people were out to make a quick buck and were using every possible trick to cut corners like providing cramped accommodation (reduced billing at GMVN), skimping on food preparations, etc. This was the same story along the entire route. But I guess for the participants it was a failure on the part of Yuvashakti.

2. First look at Bliss: Day 2 started early and we had to pack our haversack with minimum but all that is necessary for each of us to survive 7 days before we return to base camp. It was recommended to keep the bag below 10 kgs to avoid strain. My bag was around 8 kgs. Arrangements were made for some participants to load their bags on Khacchars (horses) who would walk with us. Most people however chose to carry their own bag. We had breakfast and set out for Taluka. This was a level walk of about 12 Kms through dense pine forests and over rivers and waterfalls. Was simply amazing. (I have a lot of pictures taken and will be putting them up for you guys to see). We stopped drinking our bottled water and started drinking from the streams. Natural mineral spring water. Crystal clear and ice cold. It was my first encounter with the splendor & beauty of the might Himalayas. After a leisurely walk we reached Taluka in about 4 1/2 hours. Its a small village with a small cricket ground and a few natives. I observed that the problem with the villages after Sankri was the lack of medical facilities. The nearest doctor is in Sankri (only approachable by foot), and for advanced medical facilities, they have to go to Dehradun (almost 300 kms and a 6 vehicle change). We had a native come over and ask if we have a doctor in our group. His wife was suffering from chronic stomach ache...Unfortunately there was no doctor in our group. Taluka and Sankri are at around 1900 meters above sea level.

3. Today was tough...with a 17 km walk with 2 very steep climbs and the air thinning slightly as we are now ascending to a height of about 2600 meters. The best part of the Himalayas is the different view and experience as you walk deeper. Each part as breathtaking as the previous. Of course lack of full commercialisation and difficulty of access has helped to keep the place clean and pollution free. Lets hope it stays that way. We walked to next base camp with 2 stops along the way for snack and lunch (we carried packed lunch). The other option is to buy food from the way side stalls setup by villagers where they serve Maggi noodles (yes the ones available in the city), eggs, roti (fresh whole wheat bread), tea etc. The walk took us almost 8 hours + the 2 hours of rest. Was gruelling and I think the longest part of the trek. We reached our base camp in a village called Seema en route via Dartmer, Basanti Nagar and Gangahut. Osla is another village on the map close to Seema but at an elevation above Seema. Osla is a much bigger village with about 50 homes. One of the amazing observations was how each home has a solar panel to capture energy into batteries and they use that to light up small lamps at night (or some of them had tape recorders). Another observation was that each family had a store house separate from their home that was snow proof, rain proof, all weather proof to store grain to last them through difficult times. These store houses are made of solid wood and have innovative locks with keys as large as my arm. The grain is kept open and not in gunny bags. Their homes consist of 2 stories, the lower one for keeping their animals (goats and cattle) and the upper one where they stay. Homes are also made of solid wood (no dearth of good wood in the Himalayas).

4. Reaching Har ki dun (Shiv ka maidan - The play ground of Lord Shiva) was the toughest part of the trek. Was a 14 km walk but the ascend of 1000 meters did the damage. The last 2-3 kms were covered stopping every 10-15 steps and then catching your breath. It took a long time to acclimatize to the thin air at 3500 meters (for first timers at this altitude). Some people even developed High altitude sickness (feverish, nausea and hallucinations). None the less it was the most beautiful spot I have ever seen. Picture perfect rivers, meadows, lots of varieties of flowers and completely untouched. This is the last point along our trek. From here we were very close to the Jamundar glacier (could see it) and the other snow capped peaks. The snow line had unfortunately receded by about 11 kms and our local guide decided against going upto there. During our last two days of walk we encountered sudden showers (In the Himalayas it can rain anytime and for any duration) with hail storms. After an overnight stay where most people couldn't sleep due to the extreme cold (around -1 deg Celsius) and the thin air...we walked to a nearby glacier and kind of relaxed on that day.

The rest of the days were going back to base camp along the same route. I strongly recommend this as a holiday for all my friends out there. The local arrangements can be done by local guides who charge about Rs. 700-900 a day for walking with you, putting you up in tents, arranging for food, et all. Be prepared to rough it out though. Definitely I will be going back for more...

Friday, April 07, 2006

Investigate all problems

I just came into office, a few days after the last release of our product, feeling relaxed and thinking about the new release. Just then my colleague, who was working on introducing a new LDAP backend, and was testing the migration code which he had written came up to me and reported a flaw in the upgrade/migrate of the just released version. As I have been sharing with you guys, we have a well defined process of making a product release with clear steps and forms which are filled up and recorded to help track back when required. I looked up the release records and found that the particular test which would have trapped this flaw was performed and the tester had reported a success on that. Surprise...Surprise...Surprise. Was it an oversight by a tester or was it a flaw in the process. So lets get to the bottom of things...

I've been very inspired by the program "Nat Geo investigates" on the National Geographic channel, where they do stories on investigations carried out for plane crashes, space shuttle crashes, and other such life threatening disasters. Last week, I saw how the investigators (combined from several countries) looked into a Swiss air crash in the Atlantic, spent over $50 million and 4 yours of painstaking work to recover the splintered millions of pieces of the aircraft to get to the bottom of the cause of the crash. Amazing...Its no wonder that the airlines industry is six sigma.

We have been following similar principals, where we analyze each issue, waste, problem and see how it can be fixed by technology, processes and training (in this order) In this particular instance, it turned out that the process form had an item to confirm that the logs are clean after upgrade....Hello...Which logs, how to check if they are clean...How much time has elapsed since the test before you check the log (logs rotate due to automated cron jobs and leave behind an empty log file which will have no errors). Improved the form immediately and retrained the QA team.

Every problem is an opportunity to improve the system which drives your business. Don't waste it.

Friday, March 31, 2006

Ideas - The light bulb...

Everybody goes through this phase (some stay in it forever) where maximum amount of time is spent in conceptualizing, ideating, and discussing the new thoughts. We as a team also used to do a lot of it previously. Undoubtedly an idea can be powerful, can provide breakthroughs, and we as humans are lucky to be the rare gifted species who can imagine things and conjure up images about the future. There was a time when just discussing an idea would provide immense excitement.

However, of late I realized that my attitude towards a good idea (or ideas in general) had become quite subdued. In my job, I am grappling on a daily basis with the just GTD (Getting things done)/execution. We learn daily about how complex it is to convert an idea to reality which is marketable, usable, scalable. For me an idea will excite me no end if I see it executed at least partially if not fully.

In my sphere of influence, I find that many people are caught up in idealogical world where they spend a lot of time thinking about the future (read dreaming), marveling at the creation of other people/companies (grass is greener on the other side of the fence), reading a lot about new technologies (mostly out of context and more than required) and doing precious little themselves to get any substantial work done.

Mind you, I am not disputing the power of an idea...but nothing can beat the power of a well executed idea. So what am I saying?
1. Think and learn in the context of the work in hand. (Lean thinking, lean learning, lean execution)
2. Execute it with demonic focus (read my previous blogs for some tips on how to do this)
3. Marvel at your own creation (Let other lazy people also marvel at your creation)

OK so lets get back to work now :-)

Tuesday, March 21, 2006

Being thorough

It was close to 10 P.M., with a release in progress. All the issues pertaining to the current development/checkin had passed the QA. But as part of the elaborate QA process, one of the elements is a smoke test of the entire product (recently introduced step, previously we would smoke only the changed areas of the code since we have a very dense automated testing system). During the 'full' smoke test the QA team started reporting issues unrelated to the current checkin and as I kept analyzing each issue to see where it originated, I discovered that these were a result of our old bad habits (quick development where the developer declares the code complete as soon as the file has been saved, cursory reviews with no review checklist/framework, hurried checkins, hurried or incidental QA with inadequate support via test cases, or automation etc.).

Of course we have come a long way and now (thanks to continuous and relentless improvements) can boast of a formal and very elaborate process for development, reviews, documentation and QA. As described in my previous blog posts, we have introduced templates (forms) for each stage and formalized the process to prevent slippages. As a result each checkin goes through heat treatment and typically results in almost no kickback in terms of reopened issues. In fact our lean production system with one piece flow allows us to make a strong release almost every week.

But last night, I realized something very important - after working with many developers during my tenure as a product manager, I can recall only a handful who were really thorough. This handful of guys, without any formal process, could produce high quality stuff, checkin all by themselves and get that code released. Today some of the reviews I do make me wonder, as to what would happen if I would let that get checked in as is (used to happen before and would cause Murphy's law to apply all the time - if anything can go wrong it will). Mind you, we are talking about really smart, intelligent and capable programmers who can design and construct good code but miss on the details/or are too distracted to be thorough in completion of their work or dont know what is completion.

I am sure if you are a product manager, project lead or development lead, you might have faced this type problem in your work...So how do we tackle this issue. I feel that being thorough is a habit/style of work that can be cultivated. Its not a genetic encoding.
For you the manager, you have to foster a culture of thoroughness:
1. Formalise the processes to define completion. Have forms which prompt and trap information which would otherwise be incidental. This makes quality consistent and not 'one of'.
2. Record the filled up forms with the checkin/release to help do post mortem analysis in case a problem is to surface later. This will help you identify process improvements, or genuine lapses in following the process.
3. Don't fix the code given to you for review - return it for rework with honest feedback to the developers on the problems, good points and recommendations. Use this as a way of mentoring. If you tend to take a shortcut and fix the code, you are actually leaving the developers with the impression that they have done a great job.
4. Stop when you face any problem and immediately fix the process, templates, infrastructure or whatever is required.
For the developer, be a hawk with an eye for detail:
1. Just coding is not enough: Please understand that just writing and compiling code successfully is not enough. You need to ensure it integrates well, all connections to other code are maintained, test cases are thorough, documentation of the design and code is done to promote easy maintenance, coding standards are followed, an honest self review is done etc.
2. Be thorough: Develop this habit (was the motto of my school) to get a grip on the details (remember god lies in the details and BTW the devil also lies in the details :-), make notes, search through code to handle changed connections (many times a product has modules/components written in different programming languages, legacy code, and a certain change may not get directly caught by a compilation e.g. dropping a database table from the storage but having the code still refer to it, will not get caught by a normal build of the code and if you are unlucky may never get caught unless it is too late).
3. Stay focused: Keep disturbances away (refer to my earlier blog posting)
4. Throw a challenge: for the lead/reviewer/qa to find bugs in your code (For this you have to be really confident...comes from following my tips above)

I hope the above tips help you grow as an individual and also contribute better to the company where you are working...All the best (P.S. I am toying with idea of rating developments and also reviews on a scale of 1-10)

Friday, March 17, 2006

Keeping disturbances handy

It was noon and I sat down to do a design/code review as part of our development process. I had just wrapped up my sundry tasks of responding to mail, checking the status of ongoing projects, going to the loo, drinking water before settling into the review. Yes, for me a review of design and code before it goes further down the pipe is the most important part of the process. (Refer to my earlier postings about the importance of reviews, or where it is cheapest to fix bugs). Just as I started reading the specifications and design, my cell rang. It was one of our deployment engineers onsite at a customer location asking for some help in using one of the new features. After I finished the call and struggled to revert my attention to the review (yes its very difficult to switch your mind amongst activities easily), one of my team members called out to me for some quick help on some design decisions (Yes I don't have a cabin and sit in the middle of my group and am easily accessible - a boon and a bane). Once done with that, I again struggled and restarted the review. But by now my initial focus was disturbed and the two discussions (during the disturbances) were coming back to me to think about. It was like not being able to work at work.

I stopped what I was doing to reflect on this. I looked around and observed what other people in my team were doing. I also walked around to other divisions to see what those teams were doing. I saw one set of people intently doing their work (coding, diagramming, documenting, invoicing, selling etc) and another set of people talking to friends, chatting with friends on IM, browsing unrelated sites, and generally appearing to wile away their time. This was of course a sampler at a given moment but I also realized that these are the same set of people who keep their "disturbances handy" like I did when I started my review. i.e. Make it easy to be disturbed ( I am assuming that they are committed to the job; there are a set of people who wait for the day to end... this article is not about them). I also realized that there are two basic types of activities one where you create (a document, code, design, review...) and another were you maintain (run a process like front desk, recruitment, etc). In the case of the former, being disturbed is sure to bring down the quality of the work since the very nature of that work demands undivided attention and laser like focus to get it done (no wonder that Bill Gates takes two weeks off by himself on an island to figure out the future of Microsoft). As for the latter, the mind is not creating and is not in a stressed state trying to retain connections, make logical links, theorize etc so it can afford to be disturbed)

My recommendations while creating:
1. Isolate yourself so you will not be disturbed during that time.
2. Schedule such work at times when you are least likely to be needed urgently.
3. Turn off the phone, email and chat pop ups since these are like the door bell (Humans are inadvertently trained to respond to the doorbell and phone ring, dropping whatever they are doing, literally).
4. Focus on the task at hand.
5. Divide your time into sections where you are available for discussions, you respond to mail, return phone calls etc and sections where you are invisible.

Even as I write this, I was disturbed thrice...hope you will excuse any break in the flow due to that :-). I am going home now to complete the review...at least there only my dog will disturb me.

Friday, February 03, 2006

How much testing is "good enough"

Yesterday I had a fruitful discussion with my colleague (Kul) on the above subject. We were debating the selection of test cases, which ones will provide maximum impact or which are not likely to occur in a normal working scenario or to ignore cases which even if they fail will not damage the integrity of the data or working. Although we didn't reach any conclusion yesterday, I reflected on it later and this is what I came up with:

I believe that if you have taken the pains to encode a behavior (and validation of data input is a behavior), then there must be a good enough reason for it. In its simplest form, it could mean that if a program accepts a domain name as a parameter, and if we pass a wrong domain name, the program will do no harm (who knows ?) but may not inform the user about it via a user friendly message. So we as programmers build in data input validation checks and report neat failure messages and stop proceeding further. BUT if we don't test this case (may occur rarely, or is not harmful as we understand), we stand a chance of hitting a potential failure (could be as trivial as an embarrassing error message or worst still some damage which wasn't envisaged.)

Testing is the science of validating ALL assumptions and behaviors of a program. If you have taken the pains to encode a behavior it needs to be tested OR else the behavior need not be encoded at all. Also remember that code is never one time; it is maintained by other programmers and a regression suite of COMPLETE COVERAGE automated suites will help the maintenance programmer make changes freely.

Hence my mantra
1. Make code easy to test. Write compact and simple code with easy constructs with only necessary checks and behavior. (In my experience I have seen that this is possible even for very complex code, if it is broken down into units. This constraint will help you write better code in fact.)
2. Write data validation test cases for invalid and valid data input covering all possible (necessary and sufficient) test cases to cover all encoded behavior. You could refer to code and its flow for designing the cases. Remember if a program accepts junk input, it is more likely to excrete junk output. Control input to control output.
3. Write test cases to check the functioning of the code.
4. Automate aggressively to allow for easy regression.
5. Put in a process to maintain the test cases when the code changes.

Happy bug free coding...

Thursday, January 05, 2006

Formal reviews

Sometimes back I wrote about informal code reviews, since that is what we were following as a practice. However I found that the number of bugs in the checked in code was not reducing and the qa team had to work hard to not just validate the product but also to locate defects. We were practicing informal reviews as a way to "speed" up the development cycle. Unfortunately the reverse was actually happening - the code was checked in rapidly but the product languished in the qa stables for longer and also carried the risk of some bugs being discovered after release. As I wrote in one of my previous blogs, it gets more expensive to fix bugs as the code goes further away from you.

However, we recently introduced a formal and detailed process for review of not just the code but the entire design, test cases, documentation guidelines, and the code (peaceful reviews in the solitude of your cabin) with a defined checklist on what all to cover, including using the features of a developer build before checkin. We have had some amazing results...did some rough measurements and found that we trap about 90% of the problems before checkin especially framework plugs missed, silly errors, code connections, basic design flaws, messages etc. This is against trapping about 50% of the problems by the older method of review. On the face of it, the process seems to take longer but actually it outputs better quality code almost right first time reducing the rework thrash. Also such detailed drill down reviews provide benefits like
- Another perspective on the design, code thus bring it closer to extreme programming.
- Helps others to understand the code/design building in the redundancy into the team.

Yes, I know what you guys are thinking...that all the software books talk about formal reviews in which the developer sends code and lets people review it or the code is reviewed by multiple people as an event rather than "by the way", so why werent we doing it earlier...Like anything else, all practices if done in a context, adapted to your specific needs and learnt the hard way :-) bring about better results. Try for yourself, it works.