Posts from November 23, 2006

Stumbled across this blog. Another young life ended prematurely. Sad. Very sad.

I had the misfortune to visit a very angry customer today. He was having problems loading customer data into Siebel. I sat him down with a nice cup of tea and invited him to tell me all about his woes.

'Well Norman - it's like this you see. We used to be able to load 400,000 contacts per hour. Now performance is absolutely abysmal and it takes 12 minutes to load just 1,000.'

'Hold on. What exactly have you changed since you last loaded 400,000 per hour.'

'Nothing. Nothing at all.'

'Hang on. When were you last able to load 400,000 per hour - exactly ?'

'Err well. It was back in February, you see.'

'And you expect me to believe that you have changed nothing since then ?'

'Yes.'

'OK. I want hard copies of all change controls raised against this system since February and access to Visual Source Safe. I also want the DBA and Siebel administrator here. Now.'

'We haven't got any change controls. We don't use configuration management and I am the DBA, Siebel Admin, systems admin, Web server guru and I also make the tea. We are a small company, you see.'

'I see. That's fine.' (sigh and roll eyes upwards). This could be a long day.

And so the detailed analysis commenced. The control file (IFB) used to load the data. The key Oracle parameters. The hardware of the various servers. All identical to the configuration used back in February.

And he did load 400,000 records per hour. He had the logs to prove it. And now he could type the data in quicker. I know because I saw it with my own eyes. A real mystery.

Siebel's data loading utility is called EIM and uses CBO so time to check the statistics. This client uses Oracle 10g so statistics are automatically gathered by default (essentially gather stale) so all tables have current statistics.

Hold on a minute. All tables apart from EIM_CONTACT and a load of others.

'Why have you dropped statistics on EIM_CONTACT ?'

'We haven't.'

'Oh yes you have.'

'Oh yeah. I remember now. I was playing with a custom script to drop stats on all empty tables. Siebel told me to do this because it affects performance (Alert 1162).'

'Oh I see. How very interesting.'

It transpires that because the interface table is (correctly) truncated post-load, the statistics subsequently got deleted by this script. Kudos to this customer though for humourously naming the script 'dropem.sql'.

Quickly compute statistics on EIM_CONTACT and indexes and performance is miraculously restored.

Well almost. The client forgot to tell me that he had dropped unused indexes on the base tables to accelerate the data load still further. The 'ddlsync' utility that synchronises the Siebel Repository with the physical Oracle schema had subsequently been run which reinstated all the indexes slowing performance down slightly. The indexes are dropped and the original throughput is restored.

Well almost. The client forgot to tell me he had enabled mobile web clients as an experiment for Olof Ericcsson (VP Sales - Scandinavia). So we disable remote transaction logging to eliminate that minor overhead. Throughput is now restored.

So I said my goodbyes and left for the long drive home.

Only, on this occasion, the journey home was very short.

On this occasion, the client happened to be sitting at my kitchen table.

On this occasion, the rack of expensive application/Web/database servers were actually all co-hosted on my very own laptop.

On this occasion, the client who made all the mistakes was me.

Three, err, lucky people won the following, err, prize at the recent Oracle Partner Network day at UKOUG in Birmingham.

3 lucky winners have won a Virgin Experience Voucher to the value of £140.

Norman. Come here. I have a crisis. Quickly.

Whats up ?

'I have a flat tyre.'

It is dark. Mainly because it is 7.30 at night. I sigh inwardly and get a torch.

'Hmm. You're right. That tyre really is flat. Didn't you notice the car pulling to the right ?'

'No. The car was driving fine. Norman Junior did say that the car was leaning and he kept falling over though.'

'Hmm.'

'Look. I really need the car for work tomorrow. Shall we just re-inflate it ?'

'No. It will just be completely flat again by morning. Just drive round to the tyre place first thing and get it replaced.

'I'm not driving on that tyre. It's on the rims and might make things worse'

'Don't worry. The tyre place is only 800 yards away. I've driven with a flat tyre on the rims before. Just drive slowly.'

'No. Let's put the spare on.'

So we get a Swiss army penknife out of the boot. The multi-purpose device includes a red warning triangle, a first aid kit, a car jack, several spanners and a foldaway picnic table. After several hours, after inadvertently firing two distress flares, we work out how to assemble the jack and extract the spare tyre.

'What does that yellow sticker say ?'

'It's a temporary tyre. You're not supposed to drive long distances using it.'

'What does that massive '80' mean ?' 'You're not supposed to drive over 80 miles per hour but that won't be a problem for you tomorrow morning.'

We locate the jacking point and replace the tyre.

'Why is this tyre so small ? Look at it compared with that one !'

'It's an emergency tyre designed just to get you to a garage. It will be fine for work in the morning and then you can get it changed later.'

'Oh. I don't know. I'm not happy driving on this. I think I will just drive round to the tyre place first thing and get it changed.'

'OK. So we just changed this tyre in the dark and rain for no reason.'

Engtech has some great references stressing the importance of the blog title when trying to attract readers.

However, this merely confirms what I already knew. A couple of weeks ago, I posted this entry which brought a trickle of traffic from TailRank. This, in turn, brought another article about the same news story to my attention.

My post was titled: '361 days to go'

The other article was titled: 'Black Cat Thunderbolt Rocket Up The Arse'

Some of us have it. Some of us don't.