Web Debugging with Fiddler

In a previous blog I mentioned the free Wireshark utility, which has been my number one debugging tool for several years.  Being able to see everything coming into and out of a PC, and having enough network background to glean the important details, has served well.  During a recent problem I needed to debug a web application that only ran using encrypted (https/ssl) communications.  Wireshark was able to show me what was happening with session setups and encryption exchanges, but all the application data was just a garble of meaningless characters.  Enter Fiddler, a free tool provided by Microsoft.  

Fiddler is a web debugging proxy that captures all http, and optionally https, data from any web application that can point to your loopback address (127.0.0.1) on port 8888 (the default).  The interface displays a list of request/response pairs and its status code on the left side of the screen and a detailed breakdown of the request and its response on the right.  There are multiple views available to visualize each request and response.  I find the Statistics view useful to see the number of bytes sent and received and its response time, and the Inspectors (Raw View) to see the gory details.  Other frequently accessed views are the Inspectors (ImageView), Filters and Timeline.  Third-party developers add more features to Fiddler and I've installed neXpert to generate detail reports with suggestions for improvements, Watcher to detect potential security issues and JavaScript Formatter to make JavaScript easier to read. 

Launching Fiddler can be via its shortcut or from Internet Explorer using the Tools ... Fiddler2 option.  Fiddler will automatically change Internet Explorer's proxy settings, which also affects all other applications, like the Google Chrome browser, that use the same settings.  Mozilla's Firefox, which does not use IE's proxy, can be controlled via an option installed in the lower right-hand corner. 

Capturing and displaying encrypted data requires changing some default options, located under Tools ... Fiddler Options ... HTTPS tab.   I suggest you read the information found at the "Learn more about HTTPS Traffic decryption and certificate errors" link.

Fiddler has a number of features, more for the professional developers and tester, that go way beyond simply displaying data.   You can set breakpoints and even fiddle (hence the name) with the data and inject your own.  But just seeing what's really going on "behind-the-scenes" can be eye-opening.

For more details, instructional videos and download, visit www.fiddler.com.

The Emotion of Technology Change

I began my career in Information Technology back at Wright State University in 1974.  I didn't know anything about computers when I started, but a friend that was at WSU at the time showed me some programs and I was attracted by the logic, their math-like quality and the promise of avoiding a bunch of annoying liberal arts classes.  But little in my college experiences prepared me for how much emotions play in the technology field, either good or bad.  I've collected a few of those experiences in this article to share with you.  And if you're ever needing an ice-breaker when in the land of techies, just ask then about their most hair-raising experience.  You're sure to hear an earful and get the conversation going.  

I'll start with the worst feeling I've ever had.  I was new with a company back in 1980 and was working hard at getting something or another to work.  I was sitting at the master console of the company mainframe and realized I needed to stop any new work from starting in the system.  My brain instantly stormed two possibilities.  I could "purge the initiators" or I could "hold the queue".  Unfortunately my fingers interpreted an unfortunate combination of the two and decided to "purge the queue".  Before I thought twice my fingers had entered the requisite command and the system began to get rid of everything waiting to be run or the output of everything already complete.  When my brain figured this out a few microseconds too late, it was beyond stopping.  My stomach fell like a lead weight and my hands wanted to reach inside the console and take it back.  It was all I could to avoid vomiting on the computer room floor.  Fortunately I worked with some experienced, and nice, folks who had "been there and done that" and cut me a break.  I just wanted to hide.

On the other side of the emotional teeter-totter, one of most positively exhilarating experiences was back in the early days of the Internet.  Like most of us I  rode the modem speed curve as fast as I could.  Loved 2400 bps (bits per second), died and went to heaven when 9600 bps became reality and dreamed of the day that the promised 56,000 bps would actually work on my home PC.  The intellect could only imagine what a really high-speed connection would be like.  My time came during a trip to the IBM Raleigh North Carolina briefing center, where I stayed at the Washington Duke hotel.  I had heard they had a T-1 (1.544 megabits per second) to the Internet and couldn't wait to plug in and take off.  Yes, it was fast.  Yes, it was awesome.  But the surprising emotion was a sudden realization of the potential of the Internet.  Anything, anywhere, and in an instant.  It was as if, in my mind, the Berlin Wall had fallen that day.  I sat in my hotel room, my world changed, forever.

Some of the best emotions are deep-rooted in our past.  In 2009, one-quarter of Americans do not have a landline for telephone service and that number is increasing very day.  But I grew up with only a landline.  To date myself, when I was growing up my family had a six-digit phone number.  Crestview-3-8-3-2, which translated to 2-7-3-8-3-2 (the "C' and "r" in Crestview translating to "2-7").  And we had heavy rotary phones that rang like church bells and all the kids (five of us) would run to answer it when it rang.  And you actually had to walk over to where the phone was to use it.  Imagine that.  With that as a background we speed ahead to last year, 2009, when my wife and I, after moving from a "real landline" to a cable-provided "fake landline" the year before, decided to "cut the cord", eliminate the cordless phone and go with our cell phones and Skype.  In our heads, not a big deal.  In the emotions of our heart, a totally different deal.  A solemn call to Time-Warner.  The removal of the phones from each level of the house.  The feeling of being cut-off from the world.  Took awhile to feel whole again.  And we do miss the loud ringing on occasion.

  

Finally, I bought my first app-phone late last year, a Motorola Droid.  I've had an iPod Touch for a couple years, like the apps, but always feeling Wi-Fi needy.  I've had a love-fest with the Motorola RAZR since they first came out and have owned three.  But the Droid, again very unexpectedly, came with a stronger emotional response.  I was connected.  Always.  I could look it up.  Anywhere.  I took my stuff with me.  Anything at all.  Two email accounts, two calendars, music, videos, Twitter, Facebook, chats and messages.  Maps, alarm clock and directions, oh my!  Excited, sure, it was a dream come true.  But the unexpected emotion that swelled up was that of being overwhelmed.  Too much.  Too often.  Out of control.  Maybe some "dial tone therapy" would help. 

The Outcome-Value Statement

This article describes a concise and effective method to communicate a project, a requirement or even an organization's purpose to multiple audiences, each listening for their part of the message.  It does this by linking the work being done to the value being delivered through its expected outcome or outcomes.    

The work being done can be the list of various projects or the major components of projects.  It can a list of requirements for a new service or a new set of rules being considered.  This is the place to begin creating the Outcome-Value statement.  Simply write them down in a list with the most significant items first.  While the list could be very lengthy, it's best to summarize enough to keep its size to ten or less.  

The next step is to write down the Values you expect to gain. These fall into three categories: cost, service and risk.   It's not time yet to link the value to the work being done.  Just list the values for the entire effort.  It's okay to make these somewhat fuzzy for the purpose of this effort.  It's not intended to replace a business case analysis.  So a statement of "reduce maintenance costs" or "improve service reliability" is sufficient. 

The third step is usually the most difficult, although it comes faster as you become familiar with the process.  This involves stating or predicting one or more of the resulting Outcomes.  An Outcome is simply how are things different.  How does a business process change?  How might the people using a new service view the difference?  Perhaps a current service is being eliminated as part of this project.  The Outcomes will typically show you where your change management issues exist or where communication to affected parties will be required.  Most importantly an Outcome is not a cost, service or risk statement.  Saving a million dollars is a Value, not an Outcome.  "Reducing product recalls" is an Outcome, not a Value.  Outcomes may not be all that exciting, for example, changing a supplier that results in a million dollar savings.  It's an exciting Value, but not an exciting Outcome from your point of view.

The final step is to assign each Value to one or more Outcomes and link each work item to one or more Outcomes.  The result should clearly show what is being done, how your world will change and why it's a good thing.

Let's use implementing an email retention policy as an example.  Prior to this project people could freely keep or delete anything in their email mailbox, but new regulations and service disruptions are requiring a change.  Let's start by writing down a fictitious, but realistic, set of proposed rules. 


     - Emails older than 90 days will be automatically deleted  
     - Each user's mailbox is capped at 250 megabytes of storage
     - Email backups tapes will be erased after 30 days
     - Users can only move emails requiring longer-term storage to the content management service
     - Automatic deletions and erasures will be halted as required by legal proceedings

Step two is listing the Values expected.  

     - Reduce disk and tape storage costs (cost)  
     - Improved email system performance (service)  
     - Fewer "smoking gun" emails being kept (risk) 
     - Compliance with court orders (service) 
     - Retaining business records and knowledge (service)

Now the hardest part, the Outcomes.  Even in this made-up example it took about ten minutes to clearly state what is different.  Looking at it from the email user's viewpoint helps see the Outcomes.

     1. Email Becomes a Transitory Tool 
     2. New Processes for Retaining Emails for Legal and Business Requirements

Putting this all together and linking the new set of rules to the Outcomes, the final Outcome-Value statement is produced.  Each Value is listed as a bulleted item after its associated Outcome and after each new rule the Outcome it supports is listed in parenthesis.  

Email Retention Outcome-Value Statement

1. Email Becomes a Transitory Tool

     > Reduce disk and tape storage costs  
     > Improved email system performance 
     > Fewer "smoking gun" emails being kept

2. New Processes for Retaining Emails for Legal and Business Requirements

     > Compliance with court orders  
     > Retaining business records and knowledge

     - Emails older than 90 days will be automatically deleted (1)
     - Each user's mailbox is capped at 250 megabytes of storage (1)
     - Email backups tapes will be erased after 30 days (1)
     - Users can only move emails requiring longer-term storage to the content management service (2)
     - Automatic deletions and erasures will be halted as required by legal proceedings (2)

My Web Site List

You might think from my earlier blogs that I don't spend all that much time on the Internet.  Nothing could be further from the truth.  I write this blog on Google Docs, so I can work on it wherever and whenever.  I have a Google Chrome browser up at all times with seven Google services always open.  I have Firefox up at all times to handle RSS feeds and Google searches.  I regularly seek information from the three sources of all knowledge (Google, Wikipedia and YouTube).  Take my browsers away and I stumble upon this Earth.  

We all have our favorite web sites and I'll share mine with you.  Most you're probably familiar with, but even one new winner is worth perusing the list.  Here goes....

• Kayak.com - I use this for all my travel planning.  It is by far the best web site I've found for quickly finding good airfares.  It works like this:  (1) pick your cities and pick your date, (2) all the data for all the flights is sent to your browser, and (3) by using check boxes, sliders and buttons you can narrow or widen your search criteria and never go back to the web site for another long search.  
• SportingNewsToday.com - Part of my morning ritual.  Make coffee, let the dog out and then read "my morning paper".  SportingNewsToday has the best layout.  Quick and easy to find just the parts you like or skim the day's issue.  More like reading "the sports page" than drilling up and down the typical hierarchical web site.
• Gutenberg.org and Librivox.org - Books in the public domain are available at Gutenberg.org.  Audiobooks in the public domain are at Librivox.org.  I particularly like Audiobooks for car commutes and have gone through most of the Sherlock Holmes and Wizard of Oz books (there are over a dozen of each).  Volunteers record the Audiobooks, but I think you'll be amazed at how well most of them do.  
• Pandora.com - There are a number of good music sites and Pandora is my favorite, although Grooveshark, Finetune and Last.fm are among the other fine choices.  You build "stations" in Pandora based on an artist or song, and it uses that to play songs by the artist and songs similar to that style.  You can rate individual songs "thumbs-up" or "thumbs-down" to further tune the station to your liking.  Even cooler running on my Droid phone in the car.  "Radio" with no commercials. Sweet.
• Cio.com - My personal choice for balanced and professional reporting of technology news and topics.  Digg, Slashdot and Lifehacker are good too, but a little too much chaff, opinion and irrelevance.
• GeekBrief.tv - Cali Lewis delivers up-to-date technology news via a 3-5 minute video several times a week.  Particularly fond of Apple computers and Japanese robots, Cali delivers a lot of information in a very short time with shots of humor.  Nice not to read for a few minutes every morning, but it's still part of "my morning paper".
• Shoeknots.com - I just had to mention this web site although it contains a single page.  One day, many years ago, I was ranting about my new pair of dress shoes.  The dang shoe strings would not stay tied.  Now this was a problem I've had all my life, but this pair of strings was really bad.  I went on the Internet to find someone who sold "slipless" shoestrings.  I would pay anything.  I found Shoeknots.com, which taught me how to properly tie shoelaces.  I had been tying "granny knots" all my life.  Now I tie "square knots".  This web site taught me to seek out the knowledge of others.  A defining moment as I truly entered the age of search.  
• Bloom County and Dilbert - At www.gocomics.com/bloomcounty and www.dilbert.com.  Gotta have my morning dose of humor.  Bloom County is still hilarious the second time around.

IT in Competition

Internal IT departments faced almost no competition in the mainframe era and most of the client/server era. This inevitably led to problems as competition was introduced and at least some backlash from their previously captive audience. This is not unique to IT people, but just look around to other examples such as the 1983 breakup of the AT&T monopoly or the introduction of significant foreign competition in the steel and automobile industries. Initial efforts attempted to put the world back to the "good old days", then moved on to casting as much fear, uncertainty and doubt on the newcomers and finally reducing prices to hold back the flood waters. None address the fundamental lack of skills and perspective required to survive.

Internal IT faces an expanding competitive threat from an increasing number of sources. These include:

• Outsourcing - In other words, buying IT from another company. The economics of this are the most puzzling to grasp. Can I really buy the same thing I've been doing myself for less, and the outsourcer still can make a 15%-20% profit? Some of this can be accomplished by larger scale, but outsourcers live with competition every day and it simply makes them better at knowing their business, their costs and your contract.
• Off-shoring - The economics are simple. We make a lot of money in the U.S. compared to most of the world, there are talented people out there and being 12,000 miles away just doesn't interfere all that much, and in certain circumstances, has a speed-to-market advantage.
• Software-as-a-Service - I want it and I want it now. Sign on the line or input your credit card and you are off to the races. No worries about upgrades, fighting for capital or waiting in the IT queue-of-death. Compelling marketing messages targeted at the business user with the checkbook and the need to solve their business problem. Match made in heaven.
• Cloud Computing - A new source of competition that's currently in the fear, uncertainty and doubt phase. But the economic case for non-production servers (about 60% of the total), short-term needs and spiking workloads are very clear and compelling. But perhaps the scariest part is that all these virtual servers will pretty much all look the same and can take advantage of continuous hardware price reductions. That Amazon small instance at 8.5 cents per hour today is likely to cost about 1 cent per hour in 5 years. Simple Moore's Law.

Can your IT department survive, and even thrive, in this competitive landscape? Yes, but not without a significant improvement in your business skills. All else being the same, you have several advantages, including:

• You can see your budget and all the line item detail
• You don't have a profit-margin to obtain and retain
• You should have better insight into your companies' priorities
• You are in a position to take more risk than an outsider
• Your company knows you and you're readily available

To leverage your advantages you need to get involved, learn more about your business, stop protecting marginal jobs and embrace the changing technology options. Develop a trust relationship with business decision-makers and deliver on your promises. Be an advocate of your companies' change efforts and get involved. In the end it's all about business.

Password Craziness

There is a light at the end of the password tunnel. The only question is when will the endless craziness of longer and more complex passwords finally be tamed, for surely, either by reason or futility, it will end.

Surely you've seen the current craze, eight character passwords containing a combination of lowercase, uppercase, numbers and special characters. Let's say for the sake of argument that this is truly needed and worth every bit of aggravation. How long will it last? The basic math says about 10 years, given that Moore's Law holds and computing gets one-half as expensive every eighteen months, and that there are about 80 possible characters to choose from when building a password. To keep the same relative immunity, in 10 years it will take a 9 character password, in 20 years a 10 character passwords, etc., until such time that users revolt, or hopefully, start to question why in this world of marvelous technological innovation they must increasingly carry the security burden.

But why wait until the fires are burning around your feet and the smoke is rising to take a fresh look at the problem and solve it sooner rather than later. A few things to consider.

• It appears that the single biggest issue is using hashes to store passwords. Then if the bad guys get the hashes, it's straight-forward to crack common passwords. If this is indeed a real problem, then fix it. Use something else. Like encryption. Duh.
• Passwords can be cracked by brute force by simply trying every possible combination. This assumes that no prevention mechanism is in place to stop this tack from being successful. Since most passwords are validated by servers, this limits the number of attempts per second to the speed of the server and the intervening network, in most cases limiting the attempts to hundreds or maybe a thousand attempts per second. Compared to the over 1 quadrillion possibilities of an 80-choice, 8-character-long password, the math says it takes over 3 million days to try them all. I'll be lucky to live 30,000 days. I'll take my chances.
• Passwords that are easy to remember are easy to guess, probably taking only a few thousand attempts via a "smart force" method. Very true, and assuming that the above server just blindly tries as fast as its little Ghz will allow, a very real threat. But since humans can't try more than once every few seconds and will undoubtedly give up after a dozen or so attempts and go find that sticky note they knew they would need someday. So why can't the server just let the user try a few times and revoke their account? That actually works well unless someone, inside or outside your company's "four walls", decides to enter your userid and a few bad passwords and lock you out of your computer. It happens, trust me on this one. The best solution is for the server to simply "slow down", ever more slowly processing new attempts. The hacker can't try any more than the user will try before giving up. This is the clever method used by Lotus Notes for years. If you're lucky enough to have Lotus Notes available, try a few bad passwords and see what happens. I promise it won't hurt.
• If the hackers know all the common passwords, why do systems allow any of them to be used? Ah, the simple questions are the best, aren't they? If "egbdflth" is not in the list, wouldn't it be just as good as "Eg^-3U8i"?
• Passwords that protect files are prone to hacking, since many copies of the file can be made and large numbers of computers can simultaneously try to guess the password. Sooner or later they will get in, and that time can be greatly lengthened by stronger passwords. Ah, we've uncovered a truly good use for strong passwords. Finally. Anyone out there do this on a regular basis?

Why force unreasonable passwords on people? In my book if it's not in the hacker's list, it's good enough. I believe that's the reason behind the lowercase, uppercase, number and special character craziness. They're just trying to get you to pick a password that's not very likely to be in the hacker's list. But that makes the password much harder to remember than it really needs to be. That in turn leads to extremely weak password reset tools which "challenge" you with questions like your mother's maiden name. Wow, that's really secure. Not.

The larger strategic challenge is designing solutions that are simple at the edge and not just passwords. Simplicity is the best security and removing the human element is essential to that design. Stop requiring users to solve our security problems and start looking for solutions that address the real problems.

My Home PCs – Part 4 – Toys for Geeks

The final installment of this four-part blog contains some utilities that most home users will never need, but I find them indispensable. With the exception of WinDirStat, these toys take a reasonable amount of technical knowledge to use, although they are unlikely to cause your PC any problems if you want to give them a whirl. If nothing else, it's interesting to run Wireshark and Process Monitor to see the sheer volume of what's going on inside your PC. It's a much busier beast than you probably think.

• Wireshark - This program captures all network data packets coming into and going out of your PC, very similar to the professional Sniffer tool. Although having a network background is useful to understand all the packet headers, it's more useful to understand how an application works to make the best use of the data captured. It's a good idea to shutdown as many applications as possible before running Wireshark to reduce the data being captured. You can download Wireshark at http://www.wireshark.org and there are some very good introductory videos and other documentation at http://www.wireshark.org/docs. You'll also be installing WinPCap, included in the Wireshark download, which is the component that interfaces between Windows and Wireshark.
• Process Monitor - This is one of many sysinternals utilities that Microsoft provides and the one I find the most useful. It shows real-time file system, registry and process activity, in short, all the stuff that's happening inside your PC at a very detailed level. The tool provides filters to reduce the flood of data it produces to a more manageable level. The download is available at http://technet.microsoft.com/en-us/sysinternals/bb842062.aspx, which includes both individual links to the different tools and a single download if you want the entire suite.
• VirtualBox - For those of us that like to try out new operating systems such as Ubuntu Linux and Google Android and want to make it painless, VirtualBox is the answer, and can be found at http://www.virtualbox.org/wiki/Downloads . This Sun product comes in two versions. VirtualBox OSE (Open Source Edition) is free for all purposes and VirtualBox is free only for personal use and product evaluation. More details can be found at http://www.virtualbox.org/wiki/Editions. Virtualbox creates a virtual environment for its guested operating systems and boots up image files in the .iso format. It also handles virtual machines packaged in the Open Virtualization Format (.ovf).
• Google Calendar Sync - In today's world of technology we have a lot of duplicate tools, one for our work life and one for our home life. But having separate tools sometimes causes issues and in my world having two calendars was particularly painful. Enter Google's free Calendar Sync tool, which can sync an Outlook calendar to a Google Calendar. I have my normal Google calendar that comes with my personal GMail account, which is my home life calendar. I have another Google Calendar, using a different account, which contains a synchronized copy of my work life calendar. I setup this second account to be viewable by my home life account and I can view both my calendars at the same time, giving me a complete view of my life. And my wife does the same, shares both of her calendars with me (and vice-versa) and I can see our combined four calendars all at the same time.
• WinDirStat - We all seem to run out of hard drive space and finding good candidates to delete or move elsewhere can be tedious. WinDirStat solves that by scanning a hard drive and building a visual, color-coded "block map" of every file where the size of each block is proportional to its size. Click on the block and that file is highlighted and its directory structure displayed. By far the easiest way to clean up a hard drive I've found. This utility can be downloaded at http://sourceforge.net/projects/windirstat.