I know there are some threads around the question of improving your ads CTR with split testing, but the question I have is how often is optimum for actually making changes to the ad.

I've had an experience of setting adwords ads to show equally and every week I would look at an ad CTR and change the worst performing ad.

The ads would sometimes be getting 1000's of impressions so I reckoned that the data Adwords was showing me was statistically significance.

At the end of a 6 week period you would expect the CTR for the adgroup to have improved. But they have'nt - in fact the CTR has actually gone down.
I can't understand this if I am continually keeping the better performing ad and testing a new ad beside it.

Any ideas?

And yes, I am aware of software solutions, is that the only way forward?

I would suggest pulling a Text Ad Performance report, this will show you if the two ads you are comparing are in the same position.

Also, it going to depends on the keywords in your adgroup. If one word is getting all the imps but a low CTR on the word, it could be lowering the overall CTR of the Group.....Usually if the keywords appear in your ad that usually helps the CTR...