Thursday, November 14, 2024

Tech Check-In - How Good is the Latest A.I.? - Part I

Introduction

During the ‘90s, all a company had to do to get funding was throw around the word “Internet.” Then the dot-com bubble burst, and the venture capital tightened up, but during the aughts a company could still generate a lot of excitement by using the word “cloud.” The effect of that word wore off by the teens, when tech companies had to toss about terms like “disrupt,” “transformation,” and “Internet of Things,” but even when used together these didn’t act like much of a magic wand. Now, in the roaring ‘20s, any company mentioning “A.I.” indicates its intent to be perceived as a cutting-edge company worthy of massive funding and universal adoration. The effect is starting to wear off, of course, since there are so many poseurs. And yet, there does seem to be something magical about A.I., and I’ve put it through its paces over the years (scroll to the bottom to see a list of my posts).

Thus far I’ve been pretty skeptical of A.I. and how much it will actually “disrupt” the workplace. But the technology is evolving rapidly, so I think it’s worthwhile to periodically check in on its progress. Since I last blogged on this topic, Google has increased its generative A.I. capabilities, and OpenAI has upgraded its ChatGPT engine from GPT-3.5 to GPT-4. At the same time, A.I. technologies are facing increasing opposition from the publishing industry. In this post I will evaluate the following:

  • Google’s AI Overview and opposition to it from web publishers, myself included
  • Why the New York Times is suing OpenAI, and my own exploration of ChatGPT’s plagiarism
  • ChatGPT’s strides in taking a position and supporting it


Google AI Overview

This year, as you’ve surely noticed yourself, Google has rolled out a new feature: it distills the search results it considers most germane in order to provide a handy summary, which it displays above the search results. This is convenient for users, but is perceived as a threat by web publishers. As noted in this New York Times article, publishing executives are “worried that the paragraphs pose a big danger to their brittle business model, by sharply reducing the amount of traffic to their sites from Google.” One executive complained that “it potentially chokes off the original creators of the content,” with Google’s generative A.I. summary replacing “the publications that they have plagiarized.”

So … is this true? In a word, yes. Leveraging my own thought leadership in the realm of spelling, I asked Google, “is it kindergartner or kindergartener?” and it came back with this:


My first issue with this response is that it’s not even accurate. Both spellings (kindergartner and kindergartener) are widely accepted, with no widespread preference for one over another.

That inaccuracy aside, my main issue with this summary is that it supersedes the search results Google used to show, which put my blog post on this topic right at the very top. My post provides a far more complete answer, as everyone who’s viewed it—over 21,000 readers as of this moment—can attest. It bothers me that new readers to this post (who continue to accumulate, with 374 in the past seven days) may decrease in number over time since this query, along with Google’s new “featured snippet” and “People also ask” features, shove the link to my post down so far on the screen you have to scroll to see it. I feel like my post, as the number one “organic search” result, deserves higher placement. People wondering about this spelling are now getting a very concise, wrong answer instead of finding their way to my blog and getting the full story (and hopefully bookmarking albertnet). It’s a lose-lose.

I’m not trying to suggest Google is uniquely culpable for regurgitating publisher’s content while depriving them of readers. ChatGPT does the same thing, which gives me a nice segue to…

Why the New York Times is suing OpenAI

The New York Times recently filed a lawsuit against OpenAI and Microsoft, accusing them of using its content without permission to train AI models, including ChatGPT. The lawsuit claims that these companies have effectively taken advantage of the Times’ significant investment in high-quality journalism, turning it into products that compete with the newspaper’s own content offerings.

The above paragraph was generated by ChatGPT. I know it may seem silly of me to provide this citation, since ChatGPT isn’t a writer per se, and since its very purpose is to create text people can pass off as their own. But I personally don’t think that’s ethical, since ChatGPT is basically lifting its content from other sources without clearly crediting them.

That is to say, I find the Times’ grievance entirely credible. Without much trouble, I was able to see for myself how ChatGPT plunders my content, thus competing with my blog, without giving me sufficient citation. After I asked ChatGPT about the spelling of kindergartner/kindergartener, I asked it, “Is there a professional writer who has been inconsistent with her spelling of this word in a major magazine?” ChatGPT gave a very general response that included the statement, “The New Yorker provides a great example,” without actually giving the example (nor its source). To pin the chatbot down, I asked, “Can you give me a precise example of a New Yorker writer who has spelled kindergartner/kindergartener inconsistently?” It replied:

Yes, there is a known example of a New Yorker writer who has been inconsistent with the spelling of “kindergartner” and “kindergartener.” Rebecca Mead, a long-time contributor to The New Yorker, has reportedly used both spellings in her articles without a consistent preference. Her variations may reflect the linguistic blend between American and British English, as she was raised in England but has worked extensively in the United States… If you’re interested in more details on how and why both spellings are acceptable, various sources like Grammarhow provide further insights on regional preferences and dictionary entries for both “kindergartner” and “kindergartener.”

In terms of sources, ChatGPT’s text response mentions only Grammarhow, implying that this was the source of its answer, which is absolutely not the case. Only if you look above the text response and click on the down-chevron symbol to expand the list of sources will you see a link to my blog post:



A lazy or hurried user—i.e., one who wouldn’t bother to un-collapse the link listing and then click both links to review the source content—would never know who bothered to do the research to come up with Ms. Mead’s variant spellings, much less her background as a writer who has lived in both the UK and the US. Furthermore, since ChatGPT is designed to furnish copy-and-paste-ready content to so-called writers, it is enabling these users to plagiarize my (copyrighted!) blog without even knowing they’re doing it. As you can imagine, I have a big problem with this.

(In case you’re wondering why the Times is suing not just OpenAI but also Microsoft, it’s because of the partnership between the two companies. Microsoft has invested about $13 billion in OpenAI and there is a strong API integration between ChatGPT and Bing. This should answer the other question you may have had, which is why in the hell ChatGPT would use Bing—I mean, Bing, for crying out loud!—instead of Google.)

ChatGPT’s rhetorical improvement

As I have blogged about before, GPT-3.5 was barely able to form an opinion, much less support it. I asked it, about two years ago, to write a short blog post about why Campagnolo Super Record bike components are better than Shimano Dura-Ace, and it duly complied, concluding that “Campagnolo Super Record’s customizable shifting, lightweight construction, and iconic design give it an edge over Shimano Dura-Ace.” But then I asked it to write a blog post about why Dura-Ace is better, and it did that, too, with its final synopsis being, “Shimano Dura-Ace’s superior electronic shifting, durability, and modern design give it an edge over Campagnolo Super Record.” ChatGPT was basically brown-nosing me, telling me what it supposed I wanted to hear. And its supporting evidence for these arguments was verbose, vague, and overall very weak.

I see huge improvement in this realm. I asked the new ChatGPT to weigh in on a debate near and dear to my heart: the Dvorak keyboard layout vs. traditional QWERTY. I challenged it to compare an anti-Dvorak article by two economists in Reason magazine to my critique of that article in these pages. ChatGPT gave a very thorough synopsis of the positions, with far greater concision than GPT-3.5 could have achieved. What’s more, it went beyond just summarizing them: it came to its own conclusion, which suggests a semblance of actual understanding of the core issue. Here is its closing paragraph:

Albert’s argument appears stronger when considering individual typing efficiency, as he directly addresses typing mechanics and user experience, whereas Margolis and Liebowitz’s argument is more focused on economic theories of adoption. While Margolis and Liebowitz provide a macro-level view, Albert’s micro-level approach may resonate more with typists who are exploring the ergonomic benefits of Dvorak. Thus, for those prioritizing typing speed and comfort, Albert’s perspective might be more compelling, even if the QWERTY layout remains dominant due to historical inertia rather than functional superiority. 

The two (dipshit) economists would really wince at this, since they were trying to deny that historical inertia is responsible for the ongoing ubiquity of QWERTY. They were attempting to discredit the Dvorak layout since their delusional economic theory depends on them refuting the assertion that inertia (aka lock-in) can trounce the actual superiority of a product. I was so chuffed with ChatGPT’s conclusion, I took a victory lap and asked it who is the better writer: that is, which article has greater rhetorical power and more style. ChatGPT responded with a nicely organized and supported essay which concluded thus:

Dana Albert has more rhetorical power for a general audience due to his humor, personal anecdotes, and engaging style. Liebowitz and Margolis provide a strong argument for those interested in the economic perspective, but Albert’s style is more entertaining and may leave a lasting impression on readers curious about the Dvorak vs. QWERTY debate.

Compared to earlier versions, the modern ChatGPT is far superior. Rather than barfing up reconstituted content from the Internet, it really does appear to be applying judgment and performing true analysis. Notwithstanding my satisfaction at having my ego stroked by this disinterested third party, I’m actually kind of frightened by how closely GPT-4’s output resembles actual human thought. Whether or not A.I. will steal all our jobs, it does appear ready to displace lesser economists.

Tune in next week…

I had originally intended to cover two more topics in this post: A.I. advances in poetry and art. Alas, I see I am out of room (or more to the point, you are out of patience) so come back next week for Part II. It will provide a thorough examination of ChatGPT’s ability to write poetry (iambic pentameter and dactylic trimeter) and to create original art per the user’s specifications. (In case you were wondering, I did use ChatGPT to create the art you see above for this post. More on that next time.) Until then, you might check out the below links for more posts on A.I.

Other albertnet posts on A.I. 

—~—~—~—~—~—~—~—~—
Email me here. For a complete index of albertnet posts, click here.
Posted by at
Labels: , , , , , , , , , , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)