Who’s fault if a D tower directs you into Bravo

I didn't see anyone say "NASA" form. So I am. I had a different scenario once, many years ago. Before moving maps were standard, I had an early yolk-mounted Garmin (still have it, somewhere!). Big bucks. I knew where I was with precision, not the norm. Flying from near Binghamton to FRG (Long Island), VFR in a 172. Note that this trip is where I learned the lesson- if able, always file IFR, even if CAVU. I had flight following. In the past, I had been cleared to JFK VOR and then east. Pretty cool to overfly. Had I known what was upcoming, I would have flown the Hudson River Corridor and 500' over the southern beach, just under the Bravo shelf. But all was going well, until I get handed off. This NY Center controller was doing every non-legal shortcut. It was like he was trying for sone high score. Firing off instructions, telling pilots to NOT acknowledge unless he requested, etc. Talking a mile per minute. It was nuts. He told me to fly 090. I was right over the Long Island Sound, for like 30 miles. JUST barely outside of the Bravo at it's apex. He asked a couple if times for my on-course heading. Once it got to 180 (yeah, it was a huge detour), he told me to proceed on course. Still, no readbacks because he said so. Within a few miles, I was inside the B. CESSNA 123X, you're inside the Bravo, I didn't clear you inside the Bravo! I was "pissed," but very calmy replied, "you told everyone no readbacks, you asked me what my on-course heading was, I replied 180, and about a minute later you told me to fly on-couse. The tapes will verify this. Yes, you never said I was specifically cleared into the Bravo, but I followed your instructions precisely. I am flying 180 to Farmingdale. What would you like me to do?" A REALLY long pause. "Um, Cessna 123X, you're cleared into the Bravo. Continue on course." "WILCO 123X." He slowed down after that. I filed a NASA report and never heard anything about the incident.