Howard Jacobson: Harold Pinter didn't get my joke, and I didn't get him – until it was too late

I listened in rapture. An observer would have picked us for master and acolyte

I have just come across a story in the papers describing my rift with Harold Pinter and its subsequent repair. This flatters me. A rift implies an earlier state of friendly relations, and until very recently – I wish now it had not been so recent – we had not by any definition of the word been friends.

My first encounter with him was at a party given by Simon Gray who, as far as I could tell, had mischievously alerted Harold to a joke I'd made about his work in one of my early novels. A couple living in Cornwall and starved of culture, but in a sort of war with what passed for culture at the time, would drive to London each weekend, go to one play after another only to walk out at the interval, unless the play was by Pinter in which case they walked out at the first pause. Trust me, it's funnier on the page. But there was no reason Pinter should think it funny in the least. And no reason – I did not suppose he was reading me – for him to have come across it. Indeed, I'm pretty sure he would not have but for Simon Gray, in a spirit of absurdist high jinks, fancying the spectacle of a couple of his friends squaring up to each other.

Pinter, anyway, approached me in what I took to be a pugilistic manner and told me who he was. I told him I knew who he was. He told me he knew what I'd been writing about him. I told him I was surprised but honoured. He told me that what he was thinking did not honour me. I told him I was honoured to be dishonoured by him. He looked me up and down as though calculating how many blows it would take to floor me. One is the answer but I didn't tell him that. And so it would have gone on had Simon Gray not interrupted us. "You two met each other then?" he asked.

Looking back I wonder whether Pinter really was being menacing to me. Could he just have been making merry? You don't always know in his plays whether one character wants to kill another or just have fun with him. That's Pinter's subject – the borderline, the uncertainty, the hair-trigger moments when joking turns to violence and vice versa. We are powder kegs and clowns, and it's hard to know which makes us more dangerous.

I wasn't a great fan of his plays. Surreal minimalism leaves me dissatisfied. I like amplitude and excess. Of course he could do hyperbole, but it was the hyperbole of less, and I'm a more man. Maybe his work rattled me. What if I was wrong? What if less was more and more was less? The other reason he rattled me was that his screenplays were so good – The Servant and the The Go-Between I loved – and he spoke so compellingly about his work in interviews, that I could see he had a genius I couldn't find in his stage plays. At the edges of my professional life he nagged away at me. I was always thinking about him. Then came a decisive incident. I was chairing a Late Show Booker Prize panel, being rude as I was hired to be, when a courier on a motorbike delivered a handwritten missive (I almost said missile) from Harold Pinter – believe me when I tell you it was still blazing hot from his hand – saying he hoped never to see me discussing literature on television again. It felt like a threat to the entire edifice that was the BBC, never mind to me.

Thereafter, when we found ourselves in the same room, we took up positions in opposite corners. Or at least I took up positions in opposite corners. When he became a more overtly political figure, railing against America, I poured scorn on him. What a waste of a linguistic gift to expend it on so banal a cause, saying exactly what men with no gift for thought or language whatsoever were saying. How could a masterly writer of ambiguities sink to the same level of crude one-note commonplace as Ken Livingstone and George Galloway? How could he bear to share the air with them, let alone a platform?

Perversely, that made me think again about his plays. Such subtlety there, such crassness on the soap box. Then a second decisive event. Quite out of the blue he approached me, again at a party, and extended a hand. "There has been a certain froideur between of us of late," he said. I didn't reply, "Froideur! Harold, it's been glacial." I simply took his hand and melted. He could do that when he bent his gaze on you, he could turn you to jelly.

I am unable to account for this sudden gesture of goodwill. Perhaps it was Simon Gray's influence. Or perhaps Harold's wife had said something. He was becoming ill, perhaps she thought he ought to be settling accounts, or needed a nice Jewish boy for a friend. It's my belief that what's unspoken in Pinter's plays – the great mysterious lacuna at their heart, the object of all those apparently motiveless interrogations – is his Jewishness. Maybe Lady Antonia thought that too and wanted him to fill the space, at least in life.

Not that we talked Jews. Or, thank God, America. What we talked on the single occasion we lunched together was literature and sex. He had invited me out to discuss my new novel which he had read in proof and was generous enough – more generous than I had ever been to him – to say he liked. The novel is about jealousy, husbands and lovers, the elusive transference of erotic power – Pinter territory, in fact. Without realising it, I'd written a Pinteresque novel, Pinteresque in subject, I mean, not in form. Less I still won't do. Or can't do.

He knew screeds of James Joyce by heart, which I listened to in wondering rapture. An observer would have picked us for master and acolyte, hero and heroiser. And so it felt. Ill and angry, he was still a great talker and, to my guilty surprise, a great listener too. His deep beautiful brown eyes swallowed me whole. I was in love. No other word for it. Bewitched. I skipped home like a boy. We'd agreed we would do it again. Then Simon Gray died and the time for conviviality passed. And now Harold. So there was and will be no second lunch and I am desperately sad about it. I would have liked more of him, not less.