Window woes

Tax season is crazy time here. Every year from January to April 15 the media relentlessly presses the message: the government has your money and you need to get it back or they government wants their share of your money. If it's the latter, you hold on to it until the last possible moment: 11:59 pm on April 15.
My misfortune was to hire a contractor who owes the government money. Of course I didn't know this when I hired him last November, when he was interested in getting the work and told me it would cost nothing to install the windows. I didn't know it until January when he installed the windows and told me his accountant said he had to buy something or he would have to pay the IRS. Taxes are paid on income earned in 2004 so it didn't register much with me anyway, what's done is done, it's good to be successful, I naively thought.
His mistake was to turn up at 1:40 pm, finish at 3:40 and expect to be paid because "it was tax day" even though he hadn't completed the work. He gave me the bill and said "your painter can sand down the skim work". Alarm bells went off so I examined the windows. Bits of insulation were hanging out of one window, another had nail holes that hadn't been filled, another shavings all over and all had uneven plaster all over the frames. I told him he wasn't finished and refused to pay him until it was as good as the work he did last time. (At $500 a window it had better be excellent work. I'd already paid him for 11 windows and he'd installed 13 so I didn't think giving him the rest of the money was an incentive for him to come back and make good.)
I tackled him about the damage on the two windows from last time that he'd promised would be taken care of and he firstly denied all memory and then when I told him that I'd contacted the wholesaler about it he got partial memory recall. He phoned the wholesaler, who is sending a rep out to look. Then he asked for $1700 for today's work as he had sorted out the problem. That is way more than we agreed for the job in the first place so I refused. I was clear: no money until the work is completed to my satisfaction; I don't know that the window people won't say he did the damage and must pay. Where would that leave me. His idea? Give him the money, and go after his licence. I don't think so.
He phoned me later, he was "hurt" I was keeping the money back. It implied I didn't trust him. That's rich coming from the man who told me he wasn't going back to one client because she was too difficult.