Also, sometimes, the contestant took too long looking for it, Marc had to tell them to continue on, wasting valuable time. Heck, as rodroddyfan said, once, the flag got stuck on the contestant's helmet and Marc had to bang on the window to tell her to go down the chute, but she didn't understand what he was trying to say, and that ate up a lot of time. They kept the Sewer Chute until the move to Florida.
Easiest: Drawbridge, Drumroll, The Tank, Suspension Bridge, Slime Canal, Low Hurdles, Moonwalk, Das Blimp, In One Ear, Umpire State Building, and Typewriter. Each were very quick to negotiate and had a straightforward objective that could buy the contestants time.
I'd say some of the hardest were:
Garbage Truck: As Robert Seidelman pointed out, when they used yellow or orange gak in 1992, it made the flag impossible to find sometimes, and tripped up quite a few course runs.
Volcano: Its previous incarnation, Mount Saint Double Dare, was relatively easy, just climb to the top of the center mountain to reach the flag, and being careful not to slip on the lava gak. But this one, to make it more challenging, they hid the flag inside one of three volcanoes. Ideally, this should have been something like Soda Jerk, but the flag blended in with the color of the lava gak, or it was quite deep inside, making contestants reach way down before they could find it.
Bridging the Gap (original Drawbridge): The wire or whatever held the door to the arched structure was so tense, that it was impossible for kids to push it down and get across. Thankfully, they replaced the wire with a chain or a looser wire to make it probably the easiest obstacle ever on the program.
Wrestle-rama: The dads on Family Double Dare had no problem with this obstacle, but for shorter players, it was almost impossible to climb up the wrestler to grab the flag. There was even one occasion where a kid contestant accidentally jumped into the ring instead of his father. Marc let the kid do the obstacle so that no time would be wasted in getting him out and getting his dad in. However, the kid took a good 20 seconds or more trying to grab the flag before Robin went in to give him a boost.
Inside Out: The Super Sloppy version of Inside Out was later improved by the time they moved to Florida; Marc said that out of the forest of balloons, you could see which one had the flag, and they just had to pick it out to pop it. Family Double Dare's version was worse. There were about seven opaque balloons and only one had the orange numbered flag. The contestants had a hard time popping even one balloon (I think the contestants were given a popping tool later on), and once, the wrong flag was found, but they didn't realize until later in the course that there was a mistake and they had to return to Inside Out, eating lots of time (Nick GaS didn't rerun this episode for some reason).
Pick-It: So many course runs were halted when contestants tried to find the flag, but couldn't.
Doggie Door: It was easy to spot the flag in the dog food when it was on Double Dare and Super Sloppy Double Dare, but when they made the dog food look like cocoa puffs in Family Double Dare, it was more difficult to find, and halted quite a few families from going on.
The Gak Factory: In Double Dare 2000, I think they originally had to wait for a handle to come their way before it could slowly take them across the obstacle to the other side, eating up lots of time if they missed it. Later, they had the contestant already at the obstacle and he would negotiate it when the previous contestant found the flag on their obstacle (similar to Gum Drop when it wasn't #1), which made it a bit easier.
Test Your Strength: Most of the time, it didn't work, and Marc only told them to go on after maybe the third attempt because too much time was wasted.
Mirror Maze: This is one of my favorite obstacles because it's challenging to try and orient yourself as you see many reflections, while trying to find the flag and get out of the maze. Later in Family Double Dare, they made a fairly challenging obstacle too challenging when they tried to hide the flag underneath objects like snakes and flies, which meant that the contestant had to spend a lot of time in the maze searching every object before the flag was found.
Buried Treasure: Originally, this was probably the hardest obstacle of all, as it was just a sandbox and the flag could be anywhere inside it. Thankfully, they put inflatable trees, snakes, and other objects and Marc noted that the flag was underneath one of them.
Shark!: The flag was deep inside the shark's mouth and blended in with other objects, making it hard to find.
