Darren Ebberon was watching the scene unfold and said it looked as though the boat had lost power and was drifting towards a coastal freighter.
"We couldn't see people on the boat," he said. "They were (drifting) pretty close to a freighter... It would have been terrifying."
Police had speculated that someone may have tried to steal the boat. After more than a two hour standoff, SWAT teams swarmed the boat and placed the suspect in custody.
"It was just a person who shouldn't have been on the boat and somehow got it out into the waterway," Sgt. Sean Whitcomb said.
After boarding the boat the man locked himself in the pilot house and refused to cooperate. The man told police he took the boat because he wanted to go to West Seattle.
Officers described the ordeal as a "pretty strange event."
The Clipper was pulled by tugboat more than a quarter mile from shore, and returned home around 12:30 p.m. to Pier 69 after the bomb squad checked the boat for hazards.
"It was just, I guess, a joy ride on his part," said President and CEO of Clipper Navigation Darrell Bryan, who was the first person to notice the boat was missing. He saw it drifting about 100 feet from the dock in a southerly direction.
"It was bizarre," said Bryan. "A number of people said they thought it was a joke when I first initiated some calls and no joke. It's just bad experience."
Bryan said despite all the security measures at the Seattle waterfront the suspect climbed through an opening in the fence area then hopped on the boat.
"I don't know where his mind was because there aren't many places to dock and maybe you can get underway without having a lot knowledge but not easy to dock a boat like that," said Bryan.
Bryan said the suspect pulled out a cleat and parted some lines when he removed the boat from the moor. "It's not a small vessel and it's not easy to start-up the vessel."
"We don't have much turnover and there's a lot of loyalty with the company so the first deal was it might have been an employee disgruntled over the years - I was glad to see it wasn't and what it doesn't explain is how the heck the guy knew what to do," said Bryan.
Investigators are working to determine how the suspect managed to board the vessel and send it drifting.
Sgt. Sean Whitcomb says the man did not have permission to be aboard the passenger ferry boat. The man was being questioned Sunday afternoon and was to be booked into King County Jail for piracy or grand theft of a vessel.
The Victoria Clipper IV provides ferry service between Seattle and Victoria, B.C and was not scheduled to sail on Sunday because of minor repairs.