Being a local, I can tell you that Robin said being dropped by England at 32 for being too old (!) seriously affected his confidence and in hindsight he probably never recovered.
Just checked and the last 6 years he has average 40, 40, 19, 21, 36 and 37.
So he was obviously still a very good player in 1998 and 1999 but for some reason it all went pearshaped for a couple of years there (possibly because he was concentrating more on the captaincy than his batting?). The new pitch at the Rose Bowl didn't help either, one of the worst in county cricket, it's stil settling down and received criticism after the ODI between South Africa and ZImbabwe there in June.
Still he bounced back well to prove he could still play and appears to have gone out at the right time, albeit with a slight whimper due to suffering with injuries this year.
From cricinfo:
"In the first Test against West Indies at Edgbaston in 1995, the first ball of the match shot over the batsman's head, eluded the wicketkeeper as well, and flew away for four byes. Poor old Jason Gallian, making his Test debut, broke his hand, and Alec Stewart was banged on the gloves so often he couldn't hold the bat on the third - and final - morning. England were demolished by an innings, but Smith top-scored twice, with 46 out of 147 and, after opening in the second-innings collapse to 89 all out, possibly the finest 41 in Test history as the West Indian fast bowlers rampaged in on a two-paced pitch of variable bounce that might have been custom-built for the Ambrose & Walsh Demolition Company.
Within a year, though, Smith was gone for good from the Test side. He was a casualty of the disappointing 1995-96 season, even though he made 66 in South Africa in what turned out to be his last Test - and 75 and 25 in his last two one-dayers, at the 1996 World Cup. Smith certainly was written off too soon, but equally after that rejection he never made the weight of runs at county level that would have demanded a recall, not in the same way as, say, those perennial in-and-outers Graeme Hick and Mark Ramprakash. Smith finished with 49 first-class centuries, while Hick has piled up more than 120 and Ramps is approaching 70."
That is typical Smith, strong against pace but perceived to have a weakness against spin. Strange then, that his best averages were recorded against Sri Lanka (67), India (63) and Pakistan (44)! Smith averaged 49 in England from 36 tests (no mean, especially considering 10 of those were against WI and another 10 against AUS). By comparison, Thorpe averages 41 in England, STewart 40, Atherton 38 and Hussain 36.
He was the 98th player to score 26,000 FC runs and sits in 9th on the Hampshire list with over 18,000.