Sounds like the driver's side front brake is dragging. That would make it pull to that side when driving, then pull to the other when braking. The rotor gets hot from the constant dragging and because of the heat that brake no longer works as well when you do step down on the pedal. Its like brake fade from over using the brakes on a steep downhill, the hotter they get the worse they work, only this is all on one side. I bet if you got out and grabbed the hub after driving a ways like this you'd get a pretty nasty burn. (Been there, done that

)
As for the exact cause, I'm not sure. It could be the piston in the caliper getting sticky for one. Once they start to overheat, the brake fluid expands and multiplies the problem. The other possibility is something wrong with the caliper slides. With the clunk you are hearing, that sorta makes sense. Could be one of the slides came out, hence the clunk. It could also just be grunge on them causing things to bind and the clunk is just a symptom.
When my 79 did this it was the caliper piston binding. The pulling to one side then the other when braking was the same, but I don't recall a clunk. In any case, pull the wheel off and look that caliper over, pull it off even if you've got the time. If it were me I'd scope for missing/damaged caliper or the sliders, if all looked well then I'd just buy the rebuild kit and try that. I'd probably also repack the wheel bearing on that side, all that heat can drive the grease out in a hurry.