Thursday 13 January 2011

How to propel your IT revenues to new heights



I found the following text in a bottle which washed up ashore. Is it true? Is it fiction? Maybe, if you dabble into the wonderful world of businness software, you might find the story mildly interesting:

One day, in F., your average IT outfit...

Concerned customer 1 : Help! Help! I can't access XYZ, the login screen says the database is corrupted. Oh my! I haven't done any backups!

Tech guy: How unfortunate, you should always backup your data, we'll se what we can do but in this situation it's not unlikely to lose at least some of your data.. Oh, wait a minute my other cell phone is ringing...

Concerned customer 2: Yo! the goddamn computer says the data is gone and that I should contact tech support, so here I am!

Tech guy: Oh hello, Mr. Manners, what where you using down there? XYZ 2.0, right?

Boss: Hey, tech guy, stop talking on the phone. I just received a call from that XYZ customer, he says that his database server is gone!


Now, XYZ is not a software produced by F., the  IT outfit. It was sold by another software house to a few of  F.'s customers. F. just installed XYZ and set SQL Server up, and those customers used some other F. software anyway, so they all called F. Usually the other software house handles all of its clients directly, but not this time.

Three, four SQL Server installations corrupting data all in the same day  is something as unlikely as winning a consolation prize at the lottery, unless a new bug came up or a new virus was in the wild, and it was neither case.

Something was definitely  stinking. I was called in to check the database status. Management Studio wasn't showing anything unusual. The databases were up and running.One could select data from every table and view, the application event log was fine, physical disks health was OK, nothing strange.

The SP hosting much of XYZ business logic were all encrypted but I already knew that. How come the login screen continued to blame a data corruption error in the database if there was effectively none?

Maybe a bug? An incompatibility of some kind? I wanted to know without calling the other guys at the other software house.

The client portion of XYZ is a .NET Windows Forms executable. It's not hard to decompile a .NET executable, especially if not obfuscated.  I used Redgate's .NET reflector, a freeware assembly decompiler: I looked for the login dialog box code.

It was a simple select statement which displayed the alarming message when a stored procedure in the database returned a particular status value.

The stored procedure was, as I said, amog the encrypted stored procedures in XYZ database. I should have really called the other SH, but then I know that SQL Server's 'encryption' is really nothing more than a simple 'xor like' obfuscation easily circumventable. It can be done by hand putting the server in single user mode with some special but standard commands, or one can use one of the many tools available.

I have one of these useful little utilities at hand so I proceeded. The stored procedure state machine only checked that the system time was distant 'n' days  from an apparently arbitrary date fixed in the code. If it was, it returned the magic value triggering the 'data corruption' message.

Editing the login stored procedure magically solved all the 'data corruption' problems at F.'s customers.

Now, remember that the same installation used by the other software house for its clients was used. The only catch is that this time F.'s tech support was called.

Presumably, every customer of the other software house experienced this 'data corruption' problem a one point in time or the other (probably not the same day for every customer, but you get the idea). I ignore if, those times, money was asked to  'save' the customer data, or if it was only a ploy to 'induce' ...service recovery fidelization.

Yet I really can't recall something so shameless in so many years of IT jobs all over the country...

No comments:

Post a Comment