Colter Bay is, of course, named for the famous mountain man John Colter, who is suspected to be the first white man to visit Yellowstone. We're not sure -- it seems pretty certain that he traveled through Yellowstone, but there's every chance that some French or Spanish explorer might have been there shortly before him, without leaving a record.
Colter was on his way back to St. Louis with Lewis and Clark when he met up with a pair of trappers; with his commanders' permission, he left the expedition and went with his new partners. That venture didn't work out, but he met up with Manuel Lisa of the Missouri Fur Company and decided to try the trading business again. It was while employed by Lisa that Colter made his journey through the northern Rockies, locating native bands and trying to persuade them to conduct business with the Missouri Co. This was in the fall and winter of 1807-08.
A few years after this trek, Colter met up with William Clark again and the two discussed his travels. Almost all we know of Colter's route is all based on this notation in the map Clark was compiling of the West:
Remember that Clark and Lewis had taken careful observations of the route of the Missouri River; it's very accurate on the map, as are the locations of its major tributaries. Lisa's fort was at the mouth of the Bighorn River, near present-day Hardin, Montana, and Colter seems to have ascended the Bighorn, turned west at the Shoshone River, and probably reached present-day Cody. Clark's map indicated "Boiling Springs" there, and although the hot springs around Cody are no longer active, there are other contemporary accounts of them; in all probability, this was the site referred to as "Colter's Hell," a name that was later - and erroneously - applied to Yellowstone itself.
That much is pretty clear from Clark's map, but after that it gets murky. Clark was filling in areas based on verbal reports, without the benefit of systematic surveys, and the results are rather imprecise, to say the least. But his map does contain two lakes, which he named "Eustis" and "Biddle." And it just happens that this area, in reality, contains two of the most prominent lakes in the Northern Rockies: Lake Yellowstone and Jackson Lake. Anchoring a reconstruction around these two landmarks, it's possible to make a plausible guess where Colter went.*
It's been proposed that he crossed over Togwootee Pass into Jackson Hole, but the map shows him skirting well south of Jackson Lake; therefore, it seems more likely that he made the difficult passage over the Wind River Range via Union Pass, between present-day Dubois and Pinedale. If that's the case, then he probably followed the Hoback River up into the south end of Jackson Hole.
It's pretty well accepted that he crossed the Tetons, probably at Teton Pass between Wilson, WY, and Victor, ID, and went north through the flat valley west of the Teton Range. The approach to "Lake Biddle" from the northwest would suggest that he crossed the northern end of the Tetons and encountered Jackson Lake for the first time.
From there, he apparently went north and found Yellowstone Lake, although it seems odd that the Snake River should be shown so truncated - it would have been the logical route all the way up to Lewis Lake or beyond. Then along the western shore of Yellowstone Lake and up the Yellowstone River - except now there are a couple more problems. The major discrepancy is that the outlet to the lake is shown to the southeast, almost 180 degrees opposite of reality. It's hard to account for an error that large, since nothing about the geography would suggest that Yellowstone Lake drains from anywhere but its northern end. Perhaps Clark simply misunderstood what he was being told.
Second, it seems surprising that Colter traveled along the west side of Yellowstone Lake without noticing the West Thumb thermal area. It would also be weird if he saw them, but didn't find them as worthy of mention as the "Boiling Springs" near Cody. Of the two possibilities, the former seems more explicable to me.
For the rest of the route, Colter is presumed to have reached Soda Butte Creek via the Lamar River, and thence to Clark's Fork and back to the Yellowstone, perhaps near Laurel. Again, it's awfully hard to make sense of Clark's map on this point. He shows Colter descending the left bank of the Yellowstone, then crossing it and, instead of following a tributary (as the Lamar is), crossing a divide into a parallel river basin. But there's really no better way of getting from Yellowstone Lake to Pryor's Fork than that, and it's pretty solid that he did travel between those two places.
If this reconstruction is accurate, this would be the approximate route that Colter took:
View Colter's route? in a larger map
*My description is largely based on the description in Mattes, Merrill J. "Behind the Legend of Colter's Hell: The Early Exploration of Yellowstone National Park," The Mississippi Valley Historical Review, Vol. 36, No. 2 (Sep., 1949), p. 254.