Wednesday, May 24, 2017

At The Beginning

At the beginning, God saw chaos in the land of computing and decided to take action. So…

On the first day, he created the System/360 mainframe and peripherals, and brought standards to the land of chaos.

On the second day, God saw the mainframe was lonely, and created RJE stations, terminals and time-sharing, bringing access to millions. And he was pleased.

On the third day, God saw that the mainframe was too large and only for the few and the rich, and he created DOS, the Intel 8080 processor and the smaller IBM PC to free the people.

On the fourth day, God saw the PC was lonely and limited, and Ethernets, file servers and the Internet were born, bringing computing freedom to the masses. And he was pleased.

On the fifth day, God saw the PC was too bulky and stationary, and created the handheld Apple Newton and Palm Pilot’s to untether the people and have computing fit in their pockets and purses.

On the sixth day, God saw the PDA was lonely and limited and created cellular data networks, which begat the iPhone and the many Android variants, and computing was carried everywhere by everyone. And he was pleased.

On the seventh day, God created billions of tiny processors running everything from cars to refrigerators and connected them to the Internet using Bluetooth, low-power RF, 5G cellular and dozens of other technologies to form the Internet of Things.

And he was pleased.

And so he rested.

But the CIO won’t get a good night’s sleep for years to come.

Monday, March 20, 2017

Google Drive Report

If you’re like me, you’ve accumulated a lot of files on your Google Drive and getting a summary might provide some helpful insights. The following Google App Script will output a summary report by Mime type (e.g. pdf, jpeg) with the total number of files and the total number of bytes used, sorted descending so the largest total number of bytes comes first. It also includes a few statistics on the Google Drive usage. You will find that certain Google file types do not count against your space quota and will have a zero file size.

To use this script...
  • Create a Google App Script (New … More … Google App Script) in your Google Drive, delete any code you see, cut and paste the code below into the script, save the script and give it a name of your choosing.
  • Create a new Google document (New … Google Docs) to hold the report, give it a name, type in a few characters (gibberish is fine) and format those characters in the font type and size you want the output report to take.  Copy (CTRL+C) the document id from the URL (the gibberish part that will look something like 1azjbDuyyT7DNvSfKsP9kOugGukF3iVN5lZ9hud47_aU), then save the document.
  • Paste (CTRL+V) the document ID into the Google App Script, replacing the bolded “YOUR FILE ID GOES HERE” on line 8.
  • Run the script (Run … AllDriveFiles or the arrowhead icon).
  • When the script is finished, you can view your report file.

The output (partial example) will look something like this.

Report generated on Tue Mar 14 2017 14:55:25 GMT-0400 (EDT)

Google Drive Storage Used is 3454522976 Bytes (3294.49 MB)
Google Drive Storage Limit is 123480309760 Bytes (117760.00 MB)
Google Drive Storage Percent Used is 2.80%

--------------------------------------------

Type = application/pdf Count = 2256 Size = 2167898741 Bytes (2067.47 MB)
Type = image/jpeg Count = 3525 Size = 1706782169 Bytes (1627.71 MB)
Type = video/mp4 Count = 8 Size = 369721056 Bytes (352.59 MB)
Type = video/quicktime Count = 3 Size = 162988797 Bytes (155.44 MB)
Type = video/x-m4v Count = 1 Size = 33350360 Bytes (31.81 MB)

If this isn’t exactly what you need, I hope it will serve as a useful starting point and reference.

----------------------------- Google App Script Code ----------------------------------------

function AllDriveFiles() {
//  
// Get date to output on the report
var today = new Date();
var todaysdate = new Date(today.getTime() - 1 * 24 * 60 * 60 * 1000);
var date = todaysdate.toDateString();
// Open the output file by its ID
var report = DocumentApp.openById('YOUR FILE ID GOES HERE');
//  
// Remove any text already in the report
report.setText("  ");
//  
// Print the date
str = 'Report generated on ' + today;
report.getBody().appendParagraph(str);
report.getBody().appendParagraph(' ');  
var used = DriveApp.getStorageUsed();
//
// Print amount of Drive space being used
str = 'Google Drive Storage Used is ' + used + ' Bytes (' + (used/1048576).toFixed(2) + ' MB)';
report.getBody().appendParagraph(str);
var limit = DriveApp.getStorageLimit();
//
// Print the total amount of Drive space
str = 'Google Drive Storage Limit is ' + limit + ' Bytes (' + (limit/1048576).toFixed(2) + ' MB)';
report.getBody().appendParagraph(str);
percent = used*100/limit;
//
// Print the percentage of the total space used by Drive - this does not include sources like GMail
str = 'Google Drive Storage Percent Used is ' + percent.toFixed(2) + '%';
report.getBody().appendParagraph(str);
report.getBody().appendParagraph(' ');
report.getBody().appendParagraph('--------------------------------------------');
report.getBody().appendParagraph(' ');
//
// Get all Drive files and store the total file count and total file space by each unique Mime Type
var arrType = [];
var arrCount = [];
var arrSize = [];
var files = DriveApp.getFiles();
while (files.hasNext()) {
 var file = files.next();
 arrLen = arrType.length;
 for (i = 0; i < arrLen; i++) {
   if (file.getMimeType() == arrType[i]) {
     arrCount[i]++;
     arrSize[i] = arrSize[i] + file.getSize();
     i = arrLen + 10;
     }
 }
 if (i == arrLen) {
    arrType[i] = file.getMimeType();
    arrCount[i] = 1;
    arrSize[i] = file.getSize();
    }
}
//
// Sort the arrays by descending total file size
arrLen = arrType.length;
var sorted=0;
var i=0;
while (sorted == 0) {
 sorted=1;
 while (i < arrLen-1) {
   if (arrSize[i] < arrSize[i+1]) {
     Type = arrType[i];
     Count = arrCount[i];
     Size = arrSize[i];
     arrType[i] = arrType[i+1];
     arrCount[i] = arrCount[i+1];
     arrSize[i] = arrSize[i+1];
     arrType[i+1] = Type;
     arrCount[i+1] = Count;
     arrSize[i+1] = Size;
     sorted=0;
     i=0;
   }
   else {
     i++;
   }
 }
}
//
// Print each detail line
arrLen = arrType.length;
for (i = 0; i < arrLen; i++) {
 str = 'Type = ' + arrType[i] + ' Count = ' + arrCount[i] + ' Size = ' + arrSize[i] + ' Bytes (' + (arrSize[i]/1048576).toFixed(2) + ' MB)';
 report.getBody().appendParagraph(str);
 }
}

Sunday, January 15, 2017

AWS IoT Button and TP-Link Smart Plug

A TP-Link smart plug is inserted into a standard electrical outlet and is controlled by the Kasa smartphone app to turn on and off the power to whatever device, for example a lamp, that’s plugged into it. It can also be controlled by voice commands when paired with an Amazon Echo. I wanted to experiment with Amazon Web Service (AWS) Internet of Things (IoT) services using my purchase of an Amazon Programmable Dash Button to control the TP-Link. The design point was that any click (single, double or long press) of the button would turn the power off if it’s currently powered on, and vice versa. That required querying the current device status, parsing the data returned and issuing the proper on/off command. Even the most simple projects, when dealing with unfamiliar technology, leads to lots of challenges and learning, which I’ll share in this blog.

I split the project in two parts, dealing first with controlling the TP-Link from a known environment, namely my MacBook Air on my home wireless network. There is no documented API for the TP-Link, but thanks to a Google search turning up a shell script on George Georgovassilis’s Techblog, I had a starting point. Commands are sent to TCP port 9999 on the TP-Link, which requires a statically-defined, internal IP address so it doesn’t move around, and I defined that on the wireless router. Executing the shell script from a Terminal prompt worked without issue. Knowing that I would be issuing those commands from the Internet (AWS) side of things and not having the ability to statically define the external IP address of my home’s Internet connection, I created a dynamic DNS name using the DYNU (www.dynu.com) service. Now when my home IP address changes, that gets sent to DYNU and they update DNS. I opened port 9999 inbound on my firewall to just the TP-Link, connected my Mac outside my home network via my Android phone’s hotspot capability. Testing was successful and the first part of the project was complete.

The second part dealt with understanding the parts and flows of AWS. When the IoT button is clicked, a message is sent to AWS which is mapped via a Simple Notification Service (SNS) message to one or more AWS Lambda functions. These functions can be written in Python 2.7 or Javascript (Node.js), but not the shell script I’d used so far. Deciding on Python, a language I had never used, I took a short crash course to learn its general syntax and converted the shell script, including updating the script’s netcat calls to standard Python socket calls. After several rounds of additional learning, I had a syntactically proper program and testing began. I was surprised to find that the data returned when querying the TP-Link was slightly different than before, and used a print statement to log the new string and modified the Python program to match. The final hurdle was learning that the socket needed to be closed and a new one created after the query and before the on/off command sent.

So now I have a working IoT button, but two factors limit its usefulness in this purpose. First, it takes about five seconds between clicking the button and the TP-Link changing its power status. Second, the IoT button is limited to about one thousand clicks before its power runs out with no way to charge or replace its battery. Turning lights on and off once a day would drain the button in a little over a year. Turning Christmas lights on and off once a day during each December would be a more suitable, and handy, use case.

Below is the code, with the only required change is updating DNS name (or IP address) in the two bolded connect statements,

AWS Lambda Python 2.7 Function

import socket
import base64
#
def lambda_handler(event, context):
  on = base64.b64decode(bytes('AAAAKtDygfiL/5r31e+UtsWg1Iv5nPCR6LfEsNGlwOLYo4HyhueT9tTu36Lfog=='))
#
  off = base64.b64decode(bytes('AAAAKtDygfiL/5r31e+UtsWg1Iv5nPCR6LfEsNGlwOLYo4HyhueT9tTu3qPeow=='))
#
  query = base64.b64decode(bytes('AAAAI9Dw0qHYq9+61/XPtJS20bTAn+yV5o/hh+jK8J7rh+vLtpbr'))
#
# Query the TP_Link for its current power status
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  s.connect(('YOUR IP OR DNS NAME', 9999))
  s.send(query)
  reply = base64.b64encode(bytes(s.recv(1024)))
  reply = reply[:7]
  s.close()
#
# If the TP_Link is off, turn it on, and vice versa
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  s.connect(('IP OR DNS NAME', 9999))
  if reply == 'AAACPND':
      s.send(on)
  else:
      s.send(off)
  s.close()

Friday, December 16, 2016

IT is Disappearing

Things disappear because either you can’t see them anymore or they become so familiar and pervasive you just totally ignore them, effectively disappearing into the background noise of life. Both of these are happening with information technology and if you’re in this field you might feel that you’re not appreciated the way you’re accustomed to, and that change is quite unexpected, given the explosion of the Internet, mobile devices, home automation technology, 4G cellular networks, driverless cars, ultra-thin laptops and hoverboards. But this isn’t a viewpoint of reality, it’s just about how people perceive the world.

Back in the “good old days”, computer rooms filled entire floors of office buildings, often viewable through thick glass windows revealing rows of big boxes with flashing lights, spinning tapes and technicians in white lab coats. Thick black coax cables connected your steel-cased terminal, with its indestructible keyboard, to secured wiring closets that routed the cables through bored out holes in floors on their way to those multi-million-dollar mainframes. You had special printers, usually large and noisy, that consumed their own special paper, boxes of continuous, green-and-white forms. If you were lucky, you had a special “knife” to separate one output from another. Those heydays changed in the late 1980’s and early 1990’s as the personal computer became popular and getting your new 486 or Pentium system with its huge boxy system unit and LCD monitor was a day to cherish. Printers morphed into sleek ink-jet and lasers, 8.5”x 11” cut-sheet wonders and loading the right drivers was a right of passage. Then starting at the end of century, things started going in reverse.

Intel-based servers, occupying a fraction of a mainframe’s space, came into fashion. That eventually led to virtualization and hyper-convergence, further shrinking its footprint until an entire room became a couple racks of equipment, hardly impressive as it stood there without a blink, a spin or a technician. Then we moved it to an outsourced data center and now we’re moving it to “the cloud”.

So servers have disappeared.

All the coax cable was removed and your office technology moved over to the same wires as your telephone. Wireless networks removed even the cord and we buried the wireless access points in the ceiling for the best coverage and even if you let its antenna hang from the ceiling, most humans don’t look up all that often, a fact I used to my advantage when hiding Easter eggs from my children. If it was above eye level, they couldn’t find it, even after they knew that’s what I was doing. Faster networks dissolved those irritating slowdowns and everyone has multi-megabit Internet at home.

So networks disappeared.

The PCs are still there, but instead of a thirty pound monster we have two pound laptops with solid-state disks to deliver ten times the performance. But we also have our smartphone and tablets almost making the need for a laptop moot. Executives routinely leave their laptop in their office while traveling, packing their instant-on, less-than-a-pound, all-day-battery iPad, most likely one they bought with their own money. The special printers are gone, consumed by the office copier. Instead of loading serious gobs of software, we access services on the Internet.

So the PCs, while not invisible yet, are fading fast.

The Help Desk has largely become faceless as tickets are entered and problems are resolved, most times without two humans conversing, much less seeing, each other. While useful, productive and necessary, it further removes the connection between people, and the human element is largely disappearing.

So the people join the PCs, not invisible totally, but again, fading.

To further compound the issue, technology is so pervasive it’s disappearing in front of our eyes. I have technology in every pocket. My car key has technology, my smartphone is technology personified, my credit cards have chips and my watch has sensors and a bluetooth connection. I have a smart TV combined with a Roku, Apple TV, Chromecast and cable box. My TV is way smarter than me. My washing machine senses its load, the dryer the its dryness and the outdoor lights sense when someone is near. The car talks to satellites, connects bluetooth and senses the key in my pocket. It was once magical, and now it’s all so familiar I take it for granted.

So it’s all around us, all the time, and we notice it about as much as the air we breathe.

It’s no wonder that the question of “What’s the value of IT” comes up more frequently. We’re disappearing and what people don’t see they naturally question. In my opinion this is not one question, but two. First portion is asking us to make visible all those things they no longer see, or at least used to see parts of. We need to realize that all these things are largely invisible these days for the variety of reasons stated above and take steps to make them real again. Something as simple as making a list of software available in your company, a major systems diagram or the number of PCs being upgraded this year can go a long way.

The second part is asking what are the people in the IT department actually doing. That question can make us very defensive, but it shouldn’t, because if we really think about it, we deliver the changes our business’s require, fix their issues and protect IT assets from a variety of threats. We need to communicate those with a non-defensive posture, using a common language and with a sense of excitement. We should take notice of whole industries that provide mostly invisible products like banking and car insurance, and learn how they convey their value to customers.

Or bury yourself in your cube with your do-not-disturb sign. Your choices will determine your fate.

Wednesday, July 20, 2016

Guessing Games

I have three simple rules for constructing passwords. They must be easy to remember, hard to guess and quick to type. Inspecting this a bit closer, the “easy” and “quick” are about me, and I can decide what is and isn’t. But the most important, “guess”, is not about me, it’s about the person trying to hack my account. So I can’t decide if my chosen password is “hard” until I get into the thoughts of the hacker. Since that is literally quite impossible, we’ll have to substitute a little research, and a little deduction, to come up with some ways they might approach their guessing. Then we can match a chosen password against these methods to see if they stand a good chance of succeeding. If they might, find another password. If they can’t, sleep easy.

For this analysis, we’ll assume the password needs to be 8 characters in length and contain at least one letter and one number. That results in 2.8 trillion possible combinations, the result of each position having 36 potential (a-z, 0-9) characters.

The most obvious method, but also the most useless, is to programmatically generate guesses. We’ll call this one “random”. Each attempt to guess has a probability of 1 divided by 2.8 trillion, or roughly ten times less likely than winning the Powerball Jackpot (1 in 292 billion) with one ticket. If your thief can try one million guesses every second, it will take about two years to try all the possibilities. So while it’s highly improbable this method could work, it’s also the only method that nobody can protect against, no matter what obscure 8 characters you pick. But this is also the method the “experts” want to make harder by making the password longer, include a capital letter or some special character. Ludacris, in my opinion. 2.8 trillion choices is tough enough.

Since most people think in similar ways, it’s not surprising that they tend to pick similar, simple passwords. We’ll call this one “popular”. Trying only a few thousand of these passwords will likely let a hacker into someone’s account. If your password looks anything even close to “password”, “12345678”, “baseball”, “football”, “superman”, “trustno1”, “sunshine”, “whatever” or “startrek”, stop reading this and change your password now. Anything you use that looks simple is a bad choice. And that rule, “not simple”, is really the only thing you really need to remember.

If a hacker is specifically targeting you, they have the time and motivation to research your life to formulate likely guesses. We’ll call this “targeted”. If you’re a big Bob Seger fan, they might try “bobseger”. If you’re Donald Trump, they might try “melania1” (his wife’s name). If you’re password is built around your personal information or interests, change it now.

The final method is based upon my observations of passwords I’ve encountered, which are very often based on a dictionary words or people’s names, followed by a number, normally “1” unless they are forced to change periodically, then between “1” and “9” (only geeks use “0”) . This final method we’ll call “dictionary”. While the English dictionary contains a little over a million words (who knew?), only 3,000 or so are commonly used. And a fraction of them are seven characters long. And while fewer in number, that’s still true for baby names. If your password starts with a 7-character name or word, again, change it now.

Based the above, and while I don’t claim it’s perfect and will always be a work in progress, it’s a good place to start, and we now have six objective tests to compare our password against. Given the way I construct my passwords, here is my score.

Easy to remember - pass
Quick to type - pass
Popular Attack - pass
Targeted Attack - pass
Dictionary Attack - pass
Random Attack - fail (but everybody fails)

So yeah, I sleep well. I hope you do too.

Sunday, June 26, 2016

The Best

According to the dictionary, “best” is defined as something that is “better than all others”. Seems simple enough and we throw the word around all the time without necessarily appreciating all the various flavors of “best” there is, and if we’re really using it properly.

At the very heart of defining “best” in more detail, I think it’s a good starting point to divide “best” into its objective meaning versus its subjective meaning. The objective form means that you can clearly state your objective, for example, that you’re looking for the most profitable solution or producing a product at the least cost. The second part is developing a means to achieve that objective that others can review and agree that it indeed results in the optimum answer. For example, at work we have a model that processes customer orders, dates needed, transportation costs and machine capacities to produce a solution that maximizes profit within those constraints. That model uses a branch of mathematics, linear programming, that will produce the optimal answer. Not saying that the inputs themselves are always right or that our model might not have room for improvement, but I can reasonably argue that we get the “best” answer. Room for debate exists, but not all that much.

On the other side of “best” is its subjective meaning, and that’s open to the wide variety of human opinion. Where is the “best” city to live? Who was the “best” basketball player ever to play the game? What is the “best” wine to serve with chicken on a cool spring evening? You have your choices, and I’ll have mine. But if you ask a million people and tally up all the results for that wine recommendation, you’re more likely to follow the advice of the crowd and not my particular taste for Viognier. I look for the crowd’s opinion of “best” all the time on amazon.com, routinely searching only for products with at least a 4-star rating. But I also read some 5-star and 1-star reviews in detail, looking for some reason a particular product would not fit my particular purpose. The power of collective opinion in helping decide “best” is most useful indeed, as it rarely steers me wrong.

But what really irritates me are people that throw around “best” and can’t explain if they’re using the objective or subjective form, or the type and breadth of peer review defined above. They simply want me to accept their “best” designation without any level of substantiation. When pressed they deliver their finest “deer-in-the-headlights” look. It would be comical if it wasn’t so sad.

So the next time you hear the term “Best practice”, just think “Their favorite pizza”, and you won’t be far off.

Wednesday, June 1, 2016

Phone Stories

In a recent blog I wrote about improving the Corporate pocket phone directory and challenging the status quo. That isn’t the only story from my 1992-1998 stint leading the Network Services group, which the phone systems group was a part. Here are a couple of my favorite phone stories.

In those days phones had a paper insert between the five left-side and five right-side sets of programmable buttons. Each button was defined a function, for example “Call Forward”, a direct dial extension for a frequent contact or other purposes. The practice had been to use a typewriter to create these inserts and cut them out to fit. But then this thing called a laser printer appeared on the scene and could be programmed to print all sorts of fonts and sizes. So I created a mainframe program that would accept ten phrases and print out large, high-quality inserts. Not always accepting of change, my folks thought I was loony and didn’t understand why I would bother. At least until the senior executives commented that they loved these new inserts because they could see them without putting on their reading glasses. It’s a great lesson in seeing a situation through your customer’s eyes instead of your own.

I inherited a large closet full of unused phones, mostly phones without the small display that could tell you who was calling you, a useful feature to make sure you answer your boss’s call but ignore the pesky salesperson. The display phones were more expensive, so the previous phone system replacement tried to keep costs down by limiting the number of people that got the nicer phone. That backfired as IT buckled to reason and pressure (mostly pressure) and purchased more display phones to replace those destined for the aforementioned closet. The phone system needed another upgrade in the mid-1990’s and I was determined to learn from past mistakes. So I simply decreed that the new system would have two phone options, a normal-size, beige display phone and a larger, beige display phone for customer service and administrative assistants. The response, from my voice folks, was adamant in that I just didn’t understand my customers. They wanted, no, demanded choices, many models and colors to choose from. Executives expected to color coordinate their phone with their office furnishings. I would be forced into the “that’s the way we’ve always done it” cornucopia of choice. But, typical of me, I insisted that I knew my customer also. First they all wanted display phones, so when they would see their two choices, they would be thrilled they wouldn’t have to argue again for a display phone. The executives, in my opinion, were largely men that wore blue suits, white shirts and ties their wives picked out for them and would accept the color-neutral beige phone without a second thought. So we setup the two choices in a conference room and invited everyone in the building to come see the choices and make their decision. The result? Happy customers, much to the surprise of the IT group. If you ask what people what they want, they will come up with lots of choices. When you give them two to choose from, they will happily make that choice also. Another lesson learned. Sometimes is just better not to give people too much choice.

Phone systems are engineered, not designed, that’s very clear to me, even to this day. I have my phone number taped to my phone. An engineer would think that’s stupid, I should know my own phone number. A designer would walk around, see numbers taped to lots of phones and add that feature to the phone’s display. The designer would also have a color graphics display, not the not-quite-black on a not-quite-white background text-only display I put up with. Sadly, I think these are the same engineers that build remote controls for TV sets. It took a design-driven company to give us the phone we truly wanted, even if we didn’t know that we did. Can you imagine what the smartphone would have looked like if engineers created it? Like a really big flip phone. Thanks, Apple, for saving us from that fate.