Ever wondered which is the best generative AI tool for writing PR content?
I have just finished conducting a series of tests to determine just that and found that the post generated by Bard produced:
- Slightly better results than a post I had written from scratch.
- Significantly better results than the post by ChatGPT.
- Dramatically better results than Claude 2.
This article reveals more details about the tests and the result we weren’t expecting but upstaged the three industry-leading chatbots.
Testing the best generative AI tool for PR content
In August, Katie Delahaye Paine, CEO of Paine Publishing, and I uploaded four posts on successive Fridays to a blog, Facebook, and LinkedIn.
Each post was about our new course designed for PR and communications professionals who need to improve their metrics and want to use Google Analytics 4 (GA4) to show their value.
I wrote the first post from scratch without help from an AI chatbot. Then, I reviewed and made edits to posts on the same topic generated by:
- GPT-3.5, an AI-powered language model developed by OpenAI.
- Bard, a collaborative AI tool developed by Google.
- Claude 2, a next-generation AI assistant based on Anthropic’s research.
We used Google’s free Campaign URL Builder tool to add campaign parameters to the URLs in the call to action at the end of each post.
This enabled us to objectively measure our custom campaign in GA4 by tracking which PR content generated more new users to a landing page on Paine Publishing’s website.
But while we were analyzing the results of our tests in GA4, we also stumbled across a much bigger surprise that is over 15 times more valuable to PR and communications professionals.
Get the daily newsletter search marketers rely on.
One human-written narrative, three AI-generated posts
Before diving into the results, let’s look at how the posts used in the test were generated.
The baseline: Human-written content
First, I decided that our test needed a baseline. So, I wrote a 1,150-word chronological narrative from scratch entitled, “Why do Digital PR people need to learn about Digital Analytics?”
The body of this post includes four award-winning case studies:
- In 2005, SEO-PR and Southwest Airlines linked $2.5 million in ticket sales to four optimized press releases.
- In 2010, SEO-PR and SES Conference & Expo “put butts in seats” at a series of events during the Great Recession.
- In 2013, SEO-PR and Get City Dealz drove an 85% increase in referral traffic in February over January.
- In 2019, SEO-PR and Rutgers generated 27% of the leads for a new online master’s degree program.
Then, I published this post on my blog on a Friday afternoon and reposted it to my LinkedIn and Facebook accounts on successive Friday afternoons.
GPT-3.5 via ChatGPT
Next, I used ChatGPT powered by the GPT-3.5 model to draft a post on the same topic entitled, “Elevate Your PR Game with Google Analytics 4 (GA4).”
When I reviewed the first draft, I thought it was too chirpy. The lead read:
- “Hey fellow PR and communications enthusiasts! Are you ready to take your campaigns and programs to the next level? Say hello to your new secret weapon: Google Analytics 4 (GA4)!”
So, I revised my prompt and ChatGPT generated a 500-word listicle that provided five reasons why GA4 was “a game-changing tool.” And it produced these snappy subheads:
- Precision meets insight.
- Short-term wins, long-term growth.
- 360-degree attribution.
- Data privacy, front and center.
- A skill that sets you apart.
Now, ChatGPT had created a listicle when I had initially asked it to:
- “Write a Facebook post that explains why PR and communications professionals would benefit from learning how to use Google Analytics 4 (GA4) to measure the results of their short-term campaigns and ongoing programs.”
Even after I revised my prompt to ask the AI-powered language model developed by OpenAI to “write a LinkedIn post,” it had generated another listicle.
This may reflect what its language model was trained to produce and optimized to deliver using Reinforcement Learning with Human Feedback (RLHF).
Then, I published this post on my LinkedIn account on a Friday afternoon and reposted it to my Facebook account and blog on successive Friday afternoons.
Bard
Then, I used Bard to draft a post on the same topic, entitled, “Why should PR and communications professionals learn GA4?”
When I reviewed the draft, I was surprised it used a spiral approach to generate 550 words of helpful content.
The spiral approach is a teaching method that involves revisiting a subject or skill area at intervals, each time at a more sophisticated level.
The spiral approach used the inverted pyramid structure to place the most fundamental information in the lead paragraphs of the post:
- GA4 is the future of Google Analytics.
- GA4 is more powerful than UA.
- GA4 is more privacy-focused.
Further details were then introduced in the following paragraphs:
- GA4 can help you track the impact of your earned media.
- GA4 can help you identify your target audience.
- GA4 can help you track the success of your campaigns over time.
This indicates that Bard’s training reflects some of the new guidance that has just been incorporated in Google’s latest algorithm update, which Barry Schwartz wrote about in Google September 2023 helpful content system update rolling out.
Before this month’s update, the previous guidance had emphasized:
The updated guidance for creating helpful, reliable, people-first content has deleted the phrase “written by people.”
The updated guidance has also revised one of the questions that Google says creators can use to “gauge if the content you’re making is helpful and reliable.”
Google added “or reviewed” to the question: “Is this content written or reviewed by an expert or enthusiast who demonstrably knows the topic well?”
I added a closing paragraph and used Google’s Campaign URL Builder to add campaign parameters to the URL.
I published it on my Facebook account on a Friday afternoon and reposted it to my blog and LinkedIn account on successive Friday afternoons to compare the results.
Claude 2
Finally, I used Claude 2 to draft a post on the same topic, entitled, “Why Should PR & Comms Pros Use Google Analytics 4?”
The next-generation AI assistant, which is based on Anthropic’s research “into training helpful, honest, and harmless AI systems,” initially failed when I asked it to:
- “Write an article that’s 1,000 to 2,000 words long about why PR and communication professionals should use Google Analytics 4 (GA4) to measure the results of their campaigns and programs.”
It wrote an article that was 497 words long.
After I pointed this out, Claude 2 generated “an expanded 1,193-word version of the article.” However, it was about 550 words long.
At that point, I shifted gears and asked Claude to:
- “Write a social media post about why PR and communication professionals should use Google Analytics 4 (GA4) to measure their campaigns and programs. Use a question and answer (Q&A) format.”
The next-generation AI assistant generated a Q&A that was only 325 words long with five questions and five short answers. For example:
Now, I know that Google had reduced the visibility of content featuring Frequently Asked Questions (FAQs) in early August. But I wanted to see if Claude 2 was up to the challenge.
Like the previous two posts, I added a closing paragraph and used Google’s Campaign URL Builder to add campaign parameters to the URL.
But unlike the previous three posts, I had specified the type of content I’d asked Claude 2 to generate, and I published this post on my blog and to my LinkedIn and Facebook accounts on the final Friday afternoon in August.
So, this may have produced apples-to-oranges results.
Analyzing the results
When we analyzed the landing page report in GA4, we found what we were looking for:
- The 550-word spiral post drafted with help from Bard had generated eight new users.
- The 1,150-word chronological story written from scratch had generated seven new users.
- The 500-word listicle drafted with help from ChatGPT had generated five new users.
- The 325-word Q&A drafted with help from Claude 2 had generated one new user.
But we also noticed that direct traffic had generated 122 new users and three conversions.
I did a Google search for “link:https://painepublishing.com/training-education/ga4-for-pr-and-comms-course/” – which is an old SEO technique to find a sampling of links to any site.
I was astounded to discover that the second listing – after the landing page on Paine’s website – was a LinkedIn page titled “Katie Delahaye Paine’s Post.”
When I clicked on the image at the bottom of the short post, it took me directly to the landing page.
In other words, the content that was over 15 times more valuable than even the post generated by Bard was an image that Paine had made into a clickable link.
Now, there are various ways to add a link to an image. In Word, right-click the image and select “Link”. Then, type or paste the hyperlink address into the “Address” field.
But you should use Google’s Campaign URL Builder tool to add campaign parameters to the URL so you can use GA4 to show your contribution to the bottom line.
You may discover that a picture is worth a thousand words – even if those words are generated by an expert or enthusiast who knows the topic well or three of the best generative AI tools for writing PR content.
Contributing authors are invited to create content for Search Engine Land and are chosen for their expertise and contribution to the search community. Our contributors work under the oversight of the editorial staff and contributions are checked for quality and relevance to our readers. The opinions they express are their own.
