I had a similar experience. It turned out to be the window switch at the dash board. If I played with it I could get it to work intermittently. That's how I isolated the problem.
It may not be your problem but..... I could get the driver's window to go down but it wouldn't go up. I had to wiggle the switch and hope. I bought a used unit on eBay and had it replaced
