Google Visualisation API

This API has some excellent potential - we have recently started using Google Sites as an extranet platform, as it works well in a multi office environment and damn easy to use and update.

A nifty feature of Google Sites is that you can plug in Google gadgets anywhere throughout the site, which was nice. But now that I've seen these gadgets, it's not just nice, it's FANTASTIC! Pick a graph type, give it an XML feed and away you go.

The Gantt charts will be nifty for scheduling. I can just spit out a feed from my task system. Org charts so everyone knows their place. Gauges and pie charts and heat maps.... oh my!

Excuse me while I go and calm myself down.

Innovation with James Gosling

The Sydney Morning Herald had an excellent article this morning on James Gosling - the principal creator of Java.

The Gosling creed

• Reward risk and don't penalise failure.
• Innovation is about risk and thinking that risk and failure are good things.
• You have to learn to trust crazy people. It's the people on the edge that have the most interesting, insightful comments.
• Have confidence in yourself. People you think are oracles of wisdom - they're just making it up.
• Listen carefully, and try to figure out what the problem is to solve.

Test Driven Development

Steve from the Newcastle coders group did a informative session on test driven development on Wednesday night - and used Eclipse with JUnit to develop some unit tests. Was good to check out eclipse, and interesting to see his methods of testing.

In fact - I'm now beginning to take the following perspective...

Bad computer performance

During my illness earlier this week I decided to do some mind numbing tasks to keep me occupied - coding was giving me a headache, but my mind was still overactive so I got down to some good ol' fashion XP speed tweaks.

One performance issue that was bugging me big time was my laptop's sluggish performance using Outlook 2007 amongst other things. It was so slow it was unreasonable, especially for a core 2 duo with 2GB ram. I hadn't run a defrag since I first got the system so it was due for a reshuffle. The following quickened things up a bit (run in this order to get best results):

1. Delete all the junk. As much email as possible (my mailbox was over 2gb), all that crap freeware that I only ever used once, temp internet files, old log files, etc, etc.
   
2. Run a system wide defrag:
   (all programs > accessories > system tools > disk defragmenter)
   This will take a long time to run (e.g. 3 or 4 hours) if it hasn't been done for a while.
   This operation will most likely skip a couple of files because they are in use – mainly outlook and desktop search. After the defrag has run, run it again, but instead of doing a full defrag, just do an analyze. In the analyze feedback, take a note what files are still significantly large and fragmented.
   
3. Then use the “contig” tool to defrag individual problem files that the windows disk defragmenter can’t fix. You can get the contig tool here.
   
   http://technet.microsoft.com/en-au/sysinternals/bb897428.aspx
   
   Be sure to install it in your windows/system32 folder. It’s a command line tool that you’ll need to follow the instructions as set out in the link above.
   
   Outlook.ost and the desktop search index file are the main ones you want to defrag individually as they are huge files (my outlook file was 2.7Gb in 650 fragments. My search index file was 345Mb in over 3500 fragments). If you're a gamer who plays Guildwars or similar, running contig on the ".dat" or main game files should also speed things up. Only do this after a full system wide defrag, and you have stopped the processes in task manager (e.g. outlook.exe, etc) as the first 2 steps free up large amounts of continuous disk space needed for the larger defragmented files.
   
   More info on the contig tool here: http://whall.org/blog/2008/02/02/defraggle-rock/#more-1812
   
4. As a last resort, reformat the hard drive and reinstall the operating system. Say goodbybe to 2 days minimum to get things back the way they were.... especially when apps like Visual Studio take hours to install.
   
5. Next time you get a new computer - get a faster hard drive. Laptops especially have slower RPM hard drives to save battery life - but once alot of files get piled on the system slow down will happen. Solid state hard drives should become more mainstream this year, get one if you can live with slightly less space.
   

Firefox gaining market share

According to reports - 31.1% of the oceania region browser use is now Firefox. This and more brow raising facts here:

http://www.webappexpo.com.au/content/view/136/45/

Will be interesting to see how IE 8 goes.

Thanks to Andrew V for the link.

Data Portability

John Dyer, the author of freetextbox amongst other things, has written a nice post about the introduction to data portability. It's almost a utopian idea that you should be able to move all your data from one site to another - for example all your friends and posts from facebook into the next new social network, or all your photos from flikr to photobucket to microsoft live to who know what else.

I like the idea. It gives way for a more democratic, less commercial ideal across the web from a user's point of view, and makes companies who want your patronage work harder by ultimately making better apps.

It's a long way off, but the liquid nature of the internet could welcome it with open arms if done properly.

Charting

Google has released a nifty new API that I can see myself using quite alot - a URL based querystring API that spits out a nice chart on demand.

http://google-code-updates.blogspot.com/2007/12/embed-charts-in-webpages-with-one-of.html

cool!

Sony Reader

David Williams, a colleague from the Newcastle coders group, alerted me to the existence of these nifty devices that are dedicated to reading eBooks through his blog, Alivad.

I am a little excited about this device for these reasons:

1. My wife is a book nerd (or to put more lightly - FREAK. sorry wifey! but you know you are...) We have a storage crisis at home, there are too many books and not enough space. I actually gave a station wagon full of books to the Wallsend book exchange 2 months ago, and we still need to find more space.
2. ebooks are cheaper then real books.
3. ebooks bring up new business opportunities (for those of us that could not set up a book shop, or get a publisher)
4. The device very closely mimics a real book, but adds extra quirks of one touch page turning and other conveniences.
5. I only ever read when I go to bed. If I brought a laptop to bed to read an eBook - insomnia, which I already suffer from (along with many other unwanted side effects ;) .....) would be magnified. A Sony reader instead of a laptop would make sure I'm not checking my emails and RSS feeds at 1.00am in the morning. (although I can see the device expand in the future to have WIFI and Internet browsing capabilities which is scaring me slightly)
6. It reminds me of the resurgence in music players in the form of MP3 players - like this is kind of device (if done right... apple anyone?) will be the next must have gadget. So far Sony has the lead in their e-ink display technology - as long as they handle the sensitive DRM issue correctly.

UPDATE: 24-Nov-07

By now, most people will know about the amazon kindle, in relation to the Sony reader. Still all of my points above apply to the imminent paradigm shift in regards to books - except one thing: if the amazon kindle rules over everything else - ignore point 3. I predict the business opportunity idea will still be prevalent - but not through the kindle if amazon holds on to the publisher stigma (although it is currently necessary to keep quality reading at this stage). A community approach of quality control I believe will ultimately win out - kind of like the rating of you tube videos with comments allowed.

If a decent eBook review site was setup independent of Amazon it would be a good thing - a bit like getting away from the Murdoch's controlling the fox network and hence a large amount of their viewers opinions (please note, its election time in Australia ATM and I'm feeling political). As long as you could buy eBooks from that site too and read them..... but could you on a kindle?

Visionscape Content Management System

Finally - we have a marketing website for our content management system, Visionscape! Development on the system was always a priority over a marketing site, but our CMS is starting to mature nicely now, it's over 3 years old and still easier to use than anything else I've tried or seen (which is why I get frustrated with many of the open source solutions out there).

The site was built in about 2 days, approval and finding time for the design implemetation were the real time killers - the system does everything else in a snap. It really brings the focus back on timely approvals from management and good copywriting to get a site out the door now - there is no dev time and setting up a site takes about 2 minutes.

Version 2 will hopefully be released mid next year.

FBML

I got my first bit of facebook markup language working today in the form of a flash movie on my profile page - but according to the doco waitforclick="false" is disabled on profile pages for aesthetic reasons.... bit weird when you can put any image you want on there. I can understand why they'd restrict CSS and javascript so it doesn't become the mySpace monster, but click for flash?

The flash I have got going (after you click of course) is a nice little depth of field demo from Mr Doob. I guess I'll have to put a "click here to activate" sign.

I'm tossing up a few ideas for a facebook app, but I can't decide what yet.

Open ID

Finally we are starting to see some kind of standard rise up across the net on the subject of authentication in the form of Open ID. Some big names are starting to sign on - AOL have annouced support and apparently Microsoft has announced interoperbility with Windows Cardspace.

This looks like it could be a goer - I'll post more info once I am able to implement it somewhere.

opensource microsoft

I been left really impressed lately with microsoft's efforts at open source. Their source forge equivalent site codeplex is starting to herald some really useful and impressive projects. They're great learning aids for all the new stuff coming out in .NET 3.5 too.

One project the left me particularly interested is paint.net - a free and powerful photo editing program, that has source code included! pwhoar...

Software development - the road to sucess

from worse than failure - says it all really:

Product Management - Feature Creep

One thing I am becoming acutely aware of while developing products is that feature creep can kill a product on many levels. Equivalent to scope creep in a project, if it’s not managed carefully or specified properly, it can get seriously out of hand.

I would much rather see our team produce high quality products that focus on a few key tasks, rather than lots of features that are all half baked. If it’s the latter the product becomes -

• Frustrating to use for the customer.
• Frustrating for the support staff to train users.
• Hard for the sales staff to believe in the product, or “stand behind it” with the conviction that there is no other solution but ours.

To combat feature creep – introduction of features needs to be formailsed and signed off on in key areas in order to minimise impact. These areas include:

1. The specification stage
2. After a review during development, after a “Change Request” has been submitted by key stakeholders. If accepted (that is, signed off) with additional costs approved, it should be written back into the specification. The development team should be appropriately notified.
3. After a product has been released, once again using appropriate Change Requests.
   

Change Requests are necessary to level out those “heat of the moment” ideas that seem like a fantastic idea at the time, but when revisited may seem more like a “what was I thinking?” brain fart. They give all involved parties an excellent tool for evaluating why the change is good (or bad), and what’s needed to get it happening.

Other types of feature creep such as developer “Gold Plating” also need to be managed. If a developer wants to introduce a nifty feature they thought of over the weekend, they need to first sell it to the stakeholders rather than compromise the final product – get them excited about it, and treat it as a form of up-selling. Ultimately it’s just the software development way of saying “would you like fries with that?” with an accompanying explanation (in the form of a Change Recommendation) of why it’s good for the end user – and putting a positive spin on charging a little extra for your efforts instead of trying to fit it in to existing project requirements. In developer utopia, it would be fantastic, especially if the client always agrees wholeheartedly. In reality – developers aren’t salesmen and not always able to cope well with rejection, so this easier said than done. This is where the production team needs to develop a good relationship with their sales team; another topic altogether.

Unfortunately it all sounds very formal and time consuming, but it’s a necessary evil to ensure the quality of what you’re delivering, both to your customers (it’s usable) and to management (it’ll cost you this much time and money to change, that ok?).

Keeping it all together

A challenge I see ahead: – many products developed for specific things (hence keeping it simple) means there needs to be some level of interoperability between them all, or at least some of them. Be assured, there will be many as time passes, and you’ll need a way to get data or files from one app to another with minimal effort. This is especially important when keeping everything in the big picture.

Web services makes this conveniently possible these days, and once again Google is the poster child in this so far with their Google account as far as a good model to follow, where one account gets you into their entire product offerings. Once you’re in one of their offerings (e.g. Gmail, or Adsense, or blogger, etc.) it’s entirely focused to the task at hand – but it’s easy to get from one application to another. They’ve even gone so far now to open up an API called GData (http://code.google.com/apis/gdata/) to allow information to be moved about. In my opinion Microsoft’s “passport” was poorly implemented – but, so was their entire online product offering except for hotmail, which was bought anyway. Hopefully they get it right with Microsoft Live.

Google Docs - Presentation

Google's new powerpoint competitor is now available in google docs - here's my first one:

http://docs.google.com/Present?docid=dw9j5pb_1cchzkk&fs=true

Excellent interface as usual, one would almost go so far so say kick arse. Nifty online features that differentiate include the ability to easily share (e.g. the above link) and also discuss through an inline AJAX chat. Very cool.

Design Reviews - Don't Overdo It

I can appreciate the need for professionalism. But no matter how perfect you think the end deliverable may be, or how often it was adjusted at request of project stakeholders, the change was ultimately "a good idea at the time".

Remember though, last week's thoughts were exactly that - last week. Seeing the project implemented may change the whole perspective on things, from every one's point of view.

This article illustrates the situation perfectly:

http://www.joelonsoftware.com/items/2007/09/11.html

How would this situation be addressed in future projects? The obvious answer would be give users frequent reviews as close to the anticipated finished product as possible. No screenshots - but real, usable examples. The pitfalls - continual additions and removals will blow out a project. This is not a bad thing though as long as the client is aware of the additional resources needed to carry out changes. If the quality of the project is higher in the end, it's all worth the extra communication.

You don't want to give a user false expectation, but you also don't want to under deliver. As long as it's properly managed and expectations are addressed - everybody will come out happier.

Sortable Lists

Is there a best practice way to order items in a list? By this - I'm not talking alphabetically or date ordered, but user ordered, the way the client wants things to show. More specifically and ideally, I would really like to see a best practice .NET way of doing it. Of course it also has to be easy to implement, and universal across any list of items... in any app we've built.

The way we have been doing it the last 6 years, is we would just add an extra field to the database (e.g. pageOrder), and have the user type in a number. The lower the number, the further it gets put down the list. (SELECT * fron tblPages ORDER BY pageOrder). Too bad if the same number gets entered for 2 records. Too bad if the records are only 1 number apart as well - it usually means the whole list needs re-arranging.

Another problem with the ordering approach is that it always gets forgotten off a spec. The client doesn't notice it until they need it, and when they need it - their argument is usually "I would have thought this an obvious inclusion". As well as this, for the developer it is always an afterthought and a chore to revisit the project to put in an ordering mechanism for 1 list. Then the vicious spiral starts where the client says - "well it's done for this list, but what about these other 5 lists?".

If we had a standard ordering mechanism that could be plugged into any list easily, that was user friendly, it would make me very happy. I am impressed with what I see over at scriptaculous, where their interactive sortable lists are very impressive, but I need it to plug into a ASP.NET datagrid, which doesn't seem quite possible yet.

Data Challenge - update

The good news is - I got the data up to the website in under 1/2 an hour; 15 minutes if I dont count the process to export it out of the POS system which is out of my control. It just needs transforming now and fitting into the website, which Andrew is going to do while I take a few days off.

Bad news is - the client REALLY wants it - and wanted us to work over the weekend to deliver it sooner. It left a bit of a bitter taste in my mouth since I was up to 2am the night before and 12am the day earlier to get to this point... I was pretty happy with my achievement until the whip cracked. I guess I need to step back and de-personalise a bit once again.

Weekend work is unfortunately not possible for me to do - at least this week. Wifey made me wipe my hands of work at 5.15pm on friday since we both got 5 days striaght off to hang out with each other in an attempt to destress a bit, and she's doing a bit of whip cracking herself to make sure I don't start coding. Actually, I shouldn't even be writing about it....

Data Challenge

Over the next 2 weeks I'll be building a data transformation app that will export approximately 50,000 product records from a retail store's POS system to a website. Not hard - but there are some challenges....

Basically I'm working with very limited resources: they have a home style 1.5mb down/256k up internet connection, the website is hosted on a shared server, and the database is on a shared SQL server with probably too many other databases. A batch update is not allowed by the host, so I have to loop through each record and insert or update 50000 times without bringing the webserver or SQL server to it's knees. It also has to be done at least once a day. I'm aiming for an hour to complete the entire operation.... I reckon I could come close, but hey - hopefully it'll be better then the last guy's efforts of 9 hours.

ahhhh i love a physical challenge.

Sticky Tickets

We launched our latest project at work a couple of weeks ago - Sticky Tickets. The plug - It's an excellent little online event management tool for the masses. Any everyday Joe who is organising an event can list their event for free, and sell tickets securely.

It was a fun project to work on - we had the opportunity to put a lotta love into this site, and the focus on simplicity and ease of use makes it something hopefully the masses will pick up and run with.

check it out: http://www.stickytickets.com.au/