Data Challenge - update

The good news is - I got the data up to the website in under 1/2 an hour; 15 minutes if I dont count the process to export it out of the POS system which is out of my control. It just needs transforming now and fitting into the website, which Andrew is going to do while I take a few days off.

Bad news is - the client REALLY wants it - and wanted us to work over the weekend to deliver it sooner. It left a bit of a bitter taste in my mouth since I was up to 2am the night before and 12am the day earlier to get to this point... I was pretty happy with my achievement until the whip cracked. I guess I need to step back and de-personalise a bit once again.

Weekend work is unfortunately not possible for me to do - at least this week. Wifey made me wipe my hands of work at 5.15pm on friday since we both got 5 days striaght off to hang out with each other in an attempt to destress a bit, and she's doing a bit of whip cracking herself to make sure I don't start coding. Actually, I shouldn't even be writing about it....

Data Challenge

Over the next 2 weeks I'll be building a data transformation app that will export approximately 50,000 product records from a retail store's POS system to a website. Not hard - but there are some challenges....

Basically I'm working with very limited resources: they have a home style 1.5mb down/256k up internet connection, the website is hosted on a shared server, and the database is on a shared SQL server with probably too many other databases. A batch update is not allowed by the host, so I have to loop through each record and insert or update 50000 times without bringing the webserver or SQL server to it's knees. It also has to be done at least once a day. I'm aiming for an hour to complete the entire operation.... I reckon I could come close, but hey - hopefully it'll be better then the last guy's efforts of 9 hours.

ahhhh i love a physical challenge.

Sticky Tickets

We launched our latest project at work a couple of weeks ago - Sticky Tickets. The plug - It's an excellent little online event management tool for the masses. Any everyday Joe who is organising an event can list their event for free, and sell tickets securely.

It was a fun project to work on - we had the opportunity to put a lotta love into this site, and the focus on simplicity and ease of use makes it something hopefully the masses will pick up and run with.

check it out: http://www.stickytickets.com.au/

how to know your software project is doomed

excellent list.

http://www.codesqueeze.com/101-ways-to-know-your-software-project-is-doomed/

Ultimate PC chair

cool.... i want one. :)

Microsoft CRM

Went to the newcastle coders group last night, where Adam Cogan of SSW talked about Microsoft CRM 4 (or titan), which is in beta but gearing up for release over the next few months.

First impressions are - it's powerful, but slow. Totally web based, it surprisingly doesn't have any AJAX, and the interface can get pretty cluttered with pop ups everywhere and a not totally intuitive interface.

It's power however comes in it's customisation. The ability to define your own entities, attributes, forms, and work flows gives it huge potential to adapt to whatever you need it to do. It's also based totally on SQL Server so reporting services can be hooked in nicely. Adam, being a Microsoft regional director himself sold the product well, but the load times and confusing visual elements held him back a bit.

I liked the customisation features, but ultimately hated the interface. I still think there could be a middle ground niche market that's not M$ CRM or salesforce.com (which is also pretty pricey... 3K per year for 10 users), but takes the good features of them and makes them better then ACT! or LeGrand. What'll make such a niche product work will be that balance of extendability, ease of use and accessibility to the masses financially. Who knows... with Google in talks with Salesforce.com they might get there before everybody else.

Google Developer Day 07 Review

I came away from the GDDo7 feeling very impressed with their professionalism - I mean, geez, what other company would get James O'Loghlin of new inventors fame as a host for the day? very cool.

Google definitely seems to have caught on to the developer evangelism thing now, which Microsoft have done well in the last few years - momentum in this area is definitely hotting up. Kind of like the "teach a man how to fish" idea, but if you fish with a Google fishing rod and tackle, you (and google) can get these benefits:

• Google Gears (offline web browsing - database built into the browser. Firefox3 and google toolbar)
• API's galore: Maps, Gadgets, Adsense, Gdata (extracting and manipulating data in google spreadsheet and calendar + other up and coming google apps),
• KML (xml framework that is a presentation language to Maps and Google Earth to feed dynamic data)
• Web application toolkit

Some impressive technologies were on display, many of them were open source for customising if needed (eg. gears), and there were some cool examples of sites already using them. check out http://www.bikely.com and http://www.propertyguru.com.au for some good examples.

There was an inkling of worry that you're tying yourself to a development maintenance spiral if they continually upgrade their APIs (I mean, how do you sell this to the end client? giving the answer "because google said so" doesn't seem to give you great credibility). Alot of them also rely on Google's service offerings, rather then being able to build your own. Don't get me wrong, I don't have the resources to build my own Maps application from scratch, but there's more a feeling of a lack of an application framework (like .NET offers) - it's more at this stage like a "plug bits and pieces into your app" kind of mentality. To their creditability though, they have begun on this road with the web app toolkit (using Java to build AJAX apps) and hopefully we'll see more of this kind of thing to give developers ultimate flexibility. I just have to go and learn Java now :).

All in all, a good day was had by all (plus I've never seen so many Mac Book Pros in the one place before) - I think it was very beneficial to both Developers and Google to get important feedback, and if the relationship continues like this I can see many reasons why Microsoft should be worried.

Google Developer Day 07

Just found out, they upsized the venue - and I'm in! preview of the day here:

http://code.google.com/events/developerday/au-home.html

Will post a review after it happens.

Generation Y

This is a very interesting article on managing fresh talent who have just entered the workplace out of uni (or genY as the oldies call it these days). Even though I still feel fairly young in my career, being a KFC trained lemming, I can sympathise with what they're getting at. I feel I've worked pretty hard myself, and it gets to me a little when someone thinks they deserve everything without proving their worth. I've had a bit of experience now dealing with this kind of thing - at this stage I guess it's a mind set that both sides need to comprimise on.

Of course there's always exceptions to the rule. Make up your own mind - read about here:

http://blogs.smh.com.au/enterprise/archives/2007/05/are_geny_really_despicable.html

How good are rss feeds?

Yes, I am a late bloomer. It takes me a little while to catch on sometimes when I'm preoccupied by alot of other things. But now that I've learnt how to use the feed reader in IE7 properly - it kinda saves me heaps of time and shows me info I want when new stuff becomes available - rather then wading through masses of irrelevent stuff. Why didn't anyone tell me about this before!?!?!

management bullies

Just read an interesting blog article on microsoft's bullying management nature from an ex employee... gives a little insight into the company. It may get results, but jeez, I kinda like to have fun at work too. I guess if you've got deep pockets you can afford to be a bully.

http://blog.tomevslin.com/2007/05/microsoft_memor.html

web software: open source vs proprietary vs writing your own

The challenges I'm facing at the moment are continually changing goal posts (so preferably the solution needs to be flexible and all encompassing), not enough time to focus on one thing (solution needs to be available immediately), it has to be of the highest quality and it's gotta add to the bottom line in some way, otherwise it's a waste of time.

So I'm looking at web software options to solve a big issue I'm having at work, and being short of that magical "one program does all" utopia, what are the options available to me? Some experiences I've found so far on three options.

Open source

Pros

• Cheep cheep!
• Customisable

Cons

• It's someone elses code... is it best practice? secure? easy to maintain? easy to configure?scalable? portable? compatible with my chosen host? there's plenty of uncertainty.
• Accountability for application stuff ups?
• Does it do exactly what you want - or does it kind of.... but not quite.
• Lower quality interface and/or useability (in general... not always the case)

Proprietary

Pros

• Higher quality interface (in general... not always the case)
• Can hold vendor accountable for issues.
• Usually plenty of documentation
• Established user base

Cons

• Up front cost.
• Support costs.
• Rigid.
• Forced workflow. You have to do things their way, or it's the highway.
• There'll always be something you want in the package that isn't there.
• There'll always be something in the package you'll never use.

Writing your own

Pros

• Ultimate flexibility
• Can sell it later!

Cons

• Analysis Cost.
• Development cost. Time, effort.
• Goalposts can change, rendering some features previously developed useless.
• You're accountable. (but this could be alright if it lessens time taken to address an issue)

In summary - I'm a developer at heart, so i'd always prefer to write my own solutions if I had the time. There's something about selling (that is... getting someone else to sell it) your own work that gives me ultimate job satisfaction. Network admins, techie pros and managers will often have a different view. Unfortunately commercial reality pisses all over my idealism much of the time, and I have to bite the bullet and face facts. :)

search engine listings

nothing strikes a cord in a website owners mindset like the realisation that "oh my gosh!!!! my site doesn't show up on the first page of google!!!!!!! it's the end of the world as we know it!!!!!!". It absolutely amazes me that people think their site should magically appear where they expect it to. So here goes - my guide to simple stuff that everyone can do to improve listings.

1. Get as many other sites to link to you as possible. The higher the traffic and more popular the sites are that are linking to you, the better chance you have. Scenario: If Amazon and YouTube had permanent links to your site on their front page, you would be in a very good position. (be realistic though… you’ll never get a link from Amazon unless you pay lots and lots of money for an advert). Blogs are a great way to get other sites linking to you. Start up your own! Comment on other like minded peoples blogs (but don’t spam them) – you might be able to strike up a relationship with them so you can swap links.
2. Change the textual content of your site often, while keeping the relevance of the information. Google keeps a copy of your site on its own servers (the copy is called Google’s “cache”), and compares this cache with your actual site on a regular basis. If you have changed the content often, in the eyes of Google you are being seen as proactive in keeping information up to date. You have to ask yourself, is your site static brochure ware? Or is your site a selling tool for your business? Selling tools such as websites need constant refinement to keep information relevant and interesting for your users – otherwise, why would they come back?
3. Major keywords need to be in “heading” format (This is HTML jargon). If you want to be found by the keyword “widget”, it needs to be in a Heading 1 format on the front page of your site. You must target your major keywords and concentrate on these throughout your site.
4. Make sure your HTML page titles display appropriate text. Don’t just give a page the title “Home”, this doesn’t mean anything to someone choosing your site over another one in a search engine listing. Give the page title your business name, followed by what the page is about. E.G. “Widgets Incorporated – About Us”. Even better is to have one of your major keywords in the title.
5. Paid advertising – Pay per click campaigns such as Google Adwords do really work. You need to make the decision of whether it's right for you.

Like I said, simple stuff. There's alot more that can be done.... but I dont feel like getting into that right now. It's actually hard work to keep a site at the top - and pretty dry work at that too. I might do a post later on for the more advanced stuff if I'm in the mood.

wiki critical mass

I get the sense that wikis are starting to gain momentum now as a business tool. Doing a bit of research - it looks like wikis have actually been around since 1995 (the original wikiwikiweb... 12 years ago!!! phwoar), but it's taken some big, well publicised examples like wikipedia to throw it in the limelight.

I think the reason for the lag in adoption was because it used to be such a radical concept to let just anybody edit information on a webpage like they were writing in a word document. I mean - geez! everybody can see it!

But now attitudes have matured. The fact that intranets are a common occurrence in the workplace, blogs are everywhere, and businesses are outgrowing their first and second generation intranets, people are looking for an easier, cheaper, more efficient way to collaborate. Especially over a geographically dispersed area - and they're easily obtainable by small business.

SMH has a great article about wikis in business - working the wiki way. check it.

de-personalise

I had my monthly management mentoring meeting today - and I took away an excellent piece of advice: de-personalise.

One of my major problems at work is that I get too emotionally involved in a project. Because I've played a big part in building up most of the products and systems at work, if someone has an issue with something I tend to take it personally. Even if the problem stems from a third party plug in shortcoming like freetextbox. The problems that arise can be any number of things - mainly user initiated, but instead of trying to train users properly or trying to learn how the product works ([sarcasm]my gosh, that would take some effort on their behalf! what a concept that I could even suggest such a thing!!![/sarcasm]), people try to shift blame, ultimately ending back in my side of the court.

So I had a talk to my mentor today about this. I didn't realise this till my mentor said today - all you need to do is de-personalise yourself from it all, don't get so emotionally involved. In the end, it's just a job. By all means though, stay passionate about the job.

Go back to the system, make people use it, keep on doing what your doing.

Hearing this little piece of advice - I felt like a weight had been lifted off my shoulders, like I had one of those "moments of clarity", like I had been enlightened.

Such a trivial little thing... you might be reading this and think "big whoopdi doo". But for me - I think I just got the right piece of advice at the right time and I feel much better for everything now. It's nice to get a detached perspective - sometimes I cant see the forest for the trees, and simple advice when your bogged down can just make everything seem so much clearer.

StumbleUpon

I found this really cool tool off digg last night called StumbleUpon. It's kind of a cross between digg and the old "hot or not" website, but instead of rating humans it rates websites by use of a toolbar in the browser, with a thumbs up or thumbs down.

You get to choose your interests that you would like to stumble upon - and when you start stumbling (click the stumble button), it actually returns very high quality sites, that you would normally never find from a search engine (or something like digg for that matter). It also takes note of what you rate good or bad to return more relavant stumbles.

havva go: http://www.stumbleupon.com/

yahoo pipes - nice explanation

I like this explanation of yahoo pipes - and the concept of "treating of the web as a database". Me being a database junkie it appeals to my sense of information organising....

http://www.readwriteweb.com/archives/yahoo_pipes_web_database.php

I guess it's a search engines job to treat the web as a database - yahoo is just making it more obvious now by letting your average everyday chum to do some of their work for them - and giving them something in return.

I still havent tried it out... so I cant comment too much. You have to sign up to yet another account which is putting me off. I had a yahoo one about 6 years ago - cant remember it now or what email it was signed on as. I'm starting to get account overload.

new collaboration system - preview 1

I finalised the new data structure of the latest project at work today. Should be a great system - it will let all our clients collaborate much more efficiently, and generally let the whole process flow much more smoother.

I would classify it to be a "web two point oh" class of app - or a more preferable term for our target market: a rich user experience web application - geared towards service based businesses. It'll enable us a much higher level of customer service and give clients a better experience right through the project life cycle.

We'll use it ourselves to prove the platform - but I think it'll be a very attractive system for similar businesses.

I've still got some specifications to complete - coding not far off beginning.

media coverage

The website they're talking about in this news release:

http://ninemsn.video.msn.com/v/en-au/v.htm?g=67c5901f-a9d0-4b48-8ac6-670a89ec1864&f=39&fg=email

Made by us!

new direction

ok, this blog needs to be ramped up. at the moment it's a bit of a mish mash of ideas/interests I had 2 or 3 years ago.... some of them I still think quite applicable, but still pretty random.

Because I'm so interested and involved in web development and system development - it's really gotta be more focussed on that.

So this is my offical "change in direction" post - i want to focus more on trends, ideas, achievments, cool stuff, all web focussed. i promise i'll be more proactive in getting info up here - i want to treat it as a kind of scrap book.

cool. i'm amped. grrrrr!