So many marketers have speculated about whether they should be publishing longer posts or just more frequently. Which will result in more traffic? More leads? HubSpot took it upon themselves to find out once and for all with an epic publishing experiment that took place over the course of six months.
- How a late night Twitter conversation between HubSpot and Moz writers inspired the experiment.
- Which type of post surprised Ginny by being a poor performer.
- How their findings have changed the way that Ginny manages the editorial calendar and the content HubSpot creates.
Listen to the podcast
Mentioned in the podcast
- Quality vs. Quantity: A 6-Month Analysis of the Age-Old Blogging Debate by Ginny Soskey via HubSpot.
Read the transcript
Stephanie: Hey everyone. It’s Stephanie Saretsky here from Unbounce, and you’re listening to Call to Action, the podcast about creating better marketing experiences. Every so often you come across a blog post that makes you stop and rethink everything that you’ve been doing with your marketing strategy. You know, the kind of post that sort of blows your mind. Well, recently HubSpot published the results of an epic blog publishing experiment that took place over the course of six months. They wanted to see if they could get to the bottom of the old quantity versus quality content debate — whether it’s better to invest in more content or simply better content — and how that affects both blog traffic and lead generation over time.
Unbounce’s Content Strategist Dan Levy spoke with Ginny Soskey, Marketing Blog Editor at HubSpot about how a late-night Twitter conversation between HubSpot and Moz employees inspired the experiment and how their findings have started to transform HubSpot’s content strategy. Plus, Dan asks the question we’ve all been wondering about. What’s up with those fluffy HubSpot blog posts like “Ten Delicious Recipes You Can Make in Your Office Kitchen?”
Dan: This was a really complex and ambitious experiment that you guys ran. What were you thinking? No, I’m just kidding.
Ginny: Like why did you even do this in the first place?
Dan: What were you thinking? No, I’m actually really interested to hear the story behind it, like where did the idea come from, and what were you hoping to get out of it?
Ginny: Totally. Honestly, it started one very late night in January off of a Twitter conversation. Rand Fishkin from Moz had tweeted about us and said, “Hey, HubSpot publishes a lot of posts. I wonder why they’re publishing so much, like I’m sure they’ve tested it.” Some people at my company had gotten involved and were like, “Yeah, we’ve tested it before, but we should probably test it again.” The last time that we had tested it was mid-2013, so a lot had changed since then.
We had hired so many people on our team. Our team had tripled. We had completely redesigned our blog. We had discovered this thing, which we call internally historical optimization where we update and republish old posts and conversion optimize them in the process. We had new products that can help us track all this, and those are just a few things that had happened since then. So we thought, “Let’s take some time to figure this out.” At first I was like, “This might take me a couple weeks.” Little did I know it would take six months.
Ginny: But at the end of it, I feel like we have such a great basis for our editorial strategy, and I can actually say, “This post type works much better for us.” It’s much easier for me to give direction to our writers and anybody in the company that would like to contribute, so I’d say it’s six months well spent.
Dan: Yeah, it must give you so much more confidence in your editorial strategy that what you guys are doing, then it’s worthwhile.
Ginny: Absolutely. I think with any editorial strategy, a lot of it is based on your gut no matter how much data you have. You could have two posts that do really well. They both generate very similar levels of use and leads and whatever other metrics you’re tracking, but if you only can publish one post a day, how do you prioritize that?
Dan: There’s so much that I wanna ask you about, but first let’s talk a little bit about the metrics you guys use on your blog because deciding on those metrics is obviously something that comes first.
Dan: You wrote that the three metrics that you guys look at are views, new leads and subscribers. How did you settle on those three?
Ginny: Yeah, so our blog is a really important part of our whole business and whole funnel, so we really try to look at our blog results based on that. So as far as the business goes, we need to be able to show that we’re generating leads, so that’s first and foremost lead generation. That’s why we have that as a metric, but we also care about the really important process of getting people into the funnel in the first place, so we have to look at audience growth. You can’t have just leads, or you can’t just have audience growth, so we try to give ourselves a more holistic view by looking at views to see what kind of topics people are most interested in and what kind of formats people are most interested in.
The subscriber is just like that half step between views and leads. These are people who are just a little bit more engaged with us. They actually wanna hear from us on a consistent basis, and then lead generation – people who wanna hear from us more often than that and maybe would be interested in our software.
Dan: So a blog subscriber isn’t the same thing as a new lead. They go on separate lists, I guess.
Ginny: Oh, yeah, definitely. To become a blog subscriber, you have to only give us your e-mail.
Ginny: We don’t have to know anything else about you. A lead, on the other hand, has to fill out a form that’s generally 13 fields long.
Ginny: And it’s like that’s intentionally, right. You really wanna make sure that it’s a good fit for everybody involved if we’re gonna actually rotate that lead to sales or something like that.
Dan: Uh huh, and I imagine having all those form fields also shows that lead that you guys are serious about getting to know them and what their needs are.
Ginny: Yeah, absolutely. We don’t wanna just call you and be like, “So are you a marketer?”
Ginny: We wanna know what you’re interested in, what are your challenges, so it makes those conversations easier and more relevant for all involved.
Dan: Okay, so let’s get to the experiment itself. I don’t think we should go too much into the methodology and the other nerdy details. The post lays all that stuff out really, really well, and it’s super interesting but probably not as interesting to hear us talk about than it is to read about, so let’s cut to the chase. What was the finding? It was all about quantity versus quality, so should you create more content of a lower quality or less content of a higher quality?
Ginny: So I’m probably gonna give a cop-out answer here, but I think they’re honestly independent variables. You have to have a minimum number of posts to do anything, right. If you’re trying to grow, and you’re posting one time a week, chances are it’s probably easier to post twice a week, right, than to just increase your quality over and over and over. At a certain point you’re gonna hit this diminishing returns threshold, and that can be hard to find – we’ve spent so long trying to figure out what that “cry uncle” point is. But once you do that, then I think it’s really important that you double down and figure out exactly what’s quality for your audience. I’m gonna pause there, have a dramatic pause.
I think there’s no one thing that is a marker for quality. It’s all depending on who your audience is, how sophisticated they are, what kinda things they actually wanna hear from you. That will dictate what you believe and publish as quality, so for me at the end of this I was like, “Great, our audience wants to hear really comprehensive posts, but they also want really short-form infographics and fun – we call them “TOFU” posts, top-of-the-funnel posts. They want all of it. That’s quality to them, and that was encouraging to me to see that there’s a mixture of posts that can be deemed as quality.
Dan: It’s also so hard to isolate quality, isn’t it?
Dan: Like so often the post that you think is gonna do really, really well and people are gonna love bombs.
Dan: And maybe the one that maybe you threw together and was like, “Eh, good enough” all of a sudden goes viral or something.
Ginny: Yes, it happens all the time, and on the one hand that’s the one thing I really, really love about blogging. You never necessarily know. You’re always trying to get more data to figure out just a little bit more about what might work in the future.
Dan: And you need quantity to do that, right?
Ginny: Yeah, yeah, if you’re only publishing one post a month or one post a week, it’s really hard to get a big enough sample to be able to say definitively this works, and this doesn’t.
Dan: And it’s also less likely that that one post is gonna be a grand slam.
Dan: Well, much of your post centers around the different types of content that you publish. I wanted to zero in on a few of them specifically, but first off how did you go about developing those formats, and how do you make sure that you’re consistently creating content with those categories in mind? Does that kind of rule your editorial calendar?
Ginny: Yeah, yeah, so I’ll answer this in two parts, really, because the developing was kind of backwards, honestly. I’ve been at HubSpot for two and a half years now and it’s been mainly in the trenches. I spent a lot of my time writing. I spent a lot of my time editing and just seeing every single post that we published, so when we wanted to do this experiment, we had to retroactively categorize all these posts, and we’ve experimented a ton in the time that I’ve been here and way more before I actually arrived. We tried things — we call them “under-100 posts” — and they’re literally explaining something in under 100 words. We tried infographics and GIFs to explain certain concepts, and we kind of were trying out these different formats.
When I went to do this analysis, I looked back and saw what most often were these unifying traits between certain post types. We actually started off with only five of the ones we ended up publishing, but as we started to look through the posts, we saw we had more post formats emerge. So it really was making sure that we were doing our due diligence both in the initial categorization and then reviewing that, but it was really retroactive. We looked at what we did and what worked and worked back from there.
Dan: So you discovered post formats that you didn’t even realize you had in that process?
Ginny: Yeah, TOFU is the perfect example for this. This is not a format that we typically would build into our editorial calendar. It was something that we were like, “I wonder if people would like this topic? It’s a little bit different than what we normally do, but let’s try it.” We tried it on a more consistent basis, and we ended up having enough posts that we could use in the analysis. So we tried to go back from there. Rather than trying to shoehorn posts into formats we decided on, we worked back and worked with what we had.
Dan: Yeah, I wanted to ask you about those TOFU posts because I think HubSpot’s really known for that, well, known for the term in general and known for producing that content. I have to admit the first time we saw a post here on your blog, something like “Ten Delicious Recipes You Can Make in Your Office Kitchen,” some of us raised an eyebrow, and we’re like, “Why is that on a marketing blog?” So what did you learn about those kinds of posts and the role that they play in your marketing funnel?
Ginny: Yeah, absolutely, that’s a great question because honestly when we first talked about doing them here, we were really skeptical. We were like, “Uh, I don’t know if this is something we should be publishing. I don’t know if people are gonna like it because it’s just something that my team and I are geeking out on and thought was funny or relevant, and maybe it would totally bomb.” But the thing is we’re always experimenting, so we’d rather put a post out and have it bomb and learn from it than to not have tried at all. So we started publishing them more, and they started to really take off.
What’s interesting about these posts is they bring in people who are typically – we call it “outside our gene pool,” people who maybe have never heard of HubSpot, maybe have never heard of inbound marketing, but would be interested in hearing about how business works, how marketing works, what things might matter to marketer – the “Delicious Recipes In Your Office Kitchen You Can Make.” They’re these unifying experiences that more people have than maybe just those who blog or just those who do SEO or those who just do landing page optimization, so we wanna be able to address those needs as well because marketers are well-rounded people, and I would love people to give our readers that type of content as well.
Dan: I guess landing page optimizers and SEOs all still make lunch.
Ginny: Exactly, everybody’s gotta eat, so there’s a way for us to talk to that. We still have to find a sweet spot, right?
Dan: Uh huh.
Ginny: We can’t just be like, “Food You Can Eat” – everybody has food. A lot of people who read our blog, they’re the hacky type who probably work at startups and a lot of startups have perks and one of those might be an office kitchen with snacks, so it’s something that our readers can really relate to that it’s not related to marketing software, but it is related to our readers and something they care about.
Dan: And how did that post do in the end?
Ginny: Actually it did really well. We got lots of comments. People were like, “Oh, I need to try this.” Some of the recipes people were a little dubious about, which I totally understand. They’re office kitchen recipes you’re hacking together, not necessarily the things you wanna make for a nice meal, but it did pretty well, yeah.
Dan: Have any other marketers – besides me I guess – given you a hard time about it?
Ginny: Oh, yeah, yeah. People are like, “Really, guys? Are you really gonna do GIFs of your office kitchen recipes?” You know what, I think it’s valid feedback, but again, I’d rather experiment with it. This is probably not our best TOFU post, but it’s definitely one that I was excited to try.
Dan: Cool, yeah. Yeah, I like that attitude of let’s just try it and see what happens.
Dan: Tell me about another format of yours, promotional posts. I know as inbound marketers we typically try to stay away from being overtly promotional on our blogs, but a few of these posts have actually been pretty effective at lead generation, right?
Ginny: You’re asking me all the great questions. This is another one that we were very skeptical, and me personally, I was like, “I don’t really feel super comfortable with this,” but what got me on board was really thinking about what we were promoting. We’re not promoting our software. We’re not saying, “Hand over your credit card. Buy our software now.” It’s just letting people know about the content that we’re publishing. It’s usually about a new ebook or a tool we’ve created. It’s really just teeing you up to consume a different piece of content.
Dan: So you’re just promoting more of the thing that they’re already reading. It’s not like you’re baiting and switching them. You’re just giving them more of that.
Ginny: Yeah, exactly, and that I think the reason they are so successful from a lead generation point is because we’re very direct about what the post is about. It’s literally like, “New ebook: How to Get 100,000 Readers for Your Blog.” We’re not saying, “Here are the secrets to generating 100,000 readers,” and suddenly we’re like, “Here’s a form to download it.” We’re very overt in our title that this is a free download, this is a free tool, so people are self-selecting into these posts.
Dan: Yeah, we actually had a similar experience where when we used to release an ebook in the past, we would try to write this whole post around it. The idea was the post itself needs to be really valuable for people, so it would be like 1,00-word posts where at the end it was like, “Oh, and here’s this free ebook.” Then we realized that we were just burying the lede at that point, right?
Ginny: Yup, exactly.
Dan: You’re not doing anybody a service if you’re not clear about the thing that you’re marketing to them or that you’re giving to them for free on top of it.
Ginny: Yeah, exactly, exactly. I think we’ve continued to do these, and it’s something I will continue to keep my eye on because maybe in six months we find that we’ve hit diminishing returns with this post type, but I think if we’re being honest and saying, “This is an ebook. If you want it, download it,” and people actually do, then we’re giving people what they want.
Dan: So two of your most prolific post formats are what you call tactical and deep tactical. Can you explain the difference between those two and what you learned about how each of them performs for you guys?
Ginny: Yeah, absolutely. So our deep tactical posts are really, really comprehensive, and a one-person blog that we’ve admired a ton is Moz. They’re the perfect example of deep tactical posts.
Ginny: They have really comprehensive data. They have really actionable examples. They use quotes. They use stats. They really back up their arguments with more than just “you should try this,” and they’re also generally long.
Ginny: They’re usually like 1,500 words or above, sometimes 2,000, 3,000, 4,000, but generally they really are comprehensively covering a topic.
Dan: Probably not a coincidence that their roots are in SEO, either.
Ginny: Yeah, exactly. They’re really, really search-friendly giving search engines more opportunities to find something relevant in your post. I think tactical on the other hand, they’re a little bit more run-of-the-mill type blog posts, and that is because sometimes they’ll address topics that people just wanna know about, but they’re kinda short, just “Five Quick Tips,” that sort of thing, and sometimes they are short just because there’s not more to cover. For example, we’ll do these posts called “quick tips,” so they’ll be like, “How to Pin a Pinterest Board On Your Site,” like how to embed it. You don’t need more than 300 words on that. I don’t think anybody wants to read more than 300 words on how to actually embed a Pinterest board.
Ginny: So and that’s fine. It’s serving a very specific purpose, so that’s how I think of the difference between deep tactical and tactical, and sometimes it’s a grey area in between. It’s the nature of having a spectrum of a certain type of post, but I think their results are wildly different. The deep tactical outperforms tactical by a huge amount. The deep tactical gets 75 percent more views than the typical post that we publish, whereas tactical gets 9 percent less.
Dan: And this is the, I guess, you measure these types of posts mainly on views.
Ginny: Yeah, so views. We also looked at leads. Deep tactical and tactical did okay for leads, tactical a little bit more than deep tactical, which I think kinda speaks to the point that you talked about before, burying the lede. It’s probably just a factor of the comprehensiveness of the deep tactical. It takes a long time to get to the bottom of the post where the CTA is, and that’s okay. Over time I think that we can figure out ways to optimize those posts in a non-spammy way. I’m not too concerned because they both aren’t this huge lead generator anyway.
Dan: Well, they both serve different roles, right.
Ginny: Yes, absolutely.
Dan: One thing I find a bit surprising from your findings is that none of your posts prove to be particularly good at generating both traffic and leads. Why do you think that is?
Ginny: Yeah, oh, I wish we found that. If you find that, let me know. I’ll have to try it out.
Dan: All right.
Ginny: I started thinking about that more because ultimately that would be the ideal case, right? We care a lot about leads. We care a lot about traffic. If we could just solve for both, then maybe we could cut down on the number of posts that we publish, or there’s so much we could do if we found that golden ticket. But I think it all comes down to what people are doing when they’re going to your post. If they’re gonna become a lead, they’re in a very different mindset. They’re probably researching and voraciously looking for more information. They are willing to give over their email. They’re willing to fill out 13 field forms to give you their information and download that ebook.
They’re just in a different research mode whereas I think if you’re just viewing an article, it’s much more passive. You’re probably scanning it through, picking out a couple things that you like, hopefully in an ideal world make it all the way to the end, but you’re probably emailing it to your boss or your co-workers or your friends being like, “Hey, check this out.” You wanna look smart, you wanna look informed, you wanna share your knowledge. Those are very different behaviors and motivations, so I think for certain posts that are designed for one motivation or the other, that kind of makes sense.
Dan: No, it totally makes sense. Yeah, and it’s sort of validating that all right, I’m gonna do this type of post. I’m not gonna expect that it generates leads because the goal of this post is to get views, and another post like if it doesn’t have a crazy amount of social shares or views, that’s okay because you’ve made the decision that this is a lead generator, so I think that kinda frees you up to really focus your content on what it’s good at and what it’s meant for.
Ginny: Yeah, totally agree.
Dan: Okay, so the last type of post that I wanna ask you about is what you call editorial posts. I think it’s the kinda post, at least for me as an editor, it’s the kinda thing I geek out over, so I was a little bit crestfallen by your findings here. Can you explain what your editorial posts look like and what you learned about them?
Ginny: Yeah, this one I was sad, too. They’re really cool posts to work on, and when you’re writing a lot of very tactical, helpful content, sometimes breaking out into an editorial style post feels really freeing and exciting, and my team loves writing them. We talk and categorize editorial posts as posts that don’t really have a tactical takeaway, like you’re not gonna go execute on something after reading this post, but they often have a narrative. They tell some sort of story. They might be data-driven, telling some story using that data. Other times they’re based off of interviews or personal experiences, but they’re much more narrative-driven rather than about executing on something, but we found that they perform even worse than our tactical posts.
Dan: Oh, no.
Ginny: Yeah, they get 18 percent fewer views than our typical post, which is so unfortunate, but actually this kinda goes back to us continuing to experiment and our past experimenting. I think it really comes down to picking the right topics and not getting rid of this as a whole, but more of just identifying the topics that people actually wanna read about and would actually take the time out of their day to read something that’s not gonna directly impact them that day.
Dan: Do you think there’s a way to make these posts maybe a little more actionable like to weave a little bit more narrative or story into your tactical posts? Can you have it both ways, do you think?
Ginny: Maybe. I’d be willing to try it.
Ginny: But I wonder if it would get to be the same way where you can’t really do – or we haven’t found a way to do – lead generation and traffic generation at the same time.
Ginny: If you’re trying to do all the things, are you really gonna do that thing very well?
Dan: Yeah, and it’s something that we struggle with as well, and we’re starting to kind of experiment with the case study as format on the blog. Case studies usually make people’s eyes glaze over, but we figure if we could find a really good story that yeah, involves the customer, but it’s really about the problem that they solved and also how they did it – include a really actionable tactic that other people could learn from.
Dan: And we’re hoping maybe it can be a really good editorial post and be tactical enough for people to wanna share it and do all those other good things.
Ginny: Totally. Well, I think in that case it’s an example. People love examples, especially around hard-to-find things like landing pages, conversion rate optimization. That’s not something you can just go to somebody’s website and identify, so I think in that case we’ve done some of those posts before, and those work really well, too. So I’d love to hear how those start working out for you, too.
Dan: Cool, yeah, I’ll let you know, but anyway, enough about us. Now that you’ve run this experiment, where do you go from here? How has it changed the way you approach your editorial calendar?
Ginny: Yeah, it’s been crazy spending so much time kind of staring at our own belly buttons being like, “How are we doing?” and then having to look up and change things based off of that. I think the biggest takeaway from this research was we need to be doing way more of the deep tactical posts, we just need to. We need to devote the time to doing them, and we also should be making a concerted effort for more of the TOFU posts. I think that those two, spending more of our time doing that and less of our time on some of the tactical posts will help us grow faster. When I was designing this new editorial strategy and looking at the numbers and trying to project if we change things, what would happen, it’s always a balancing game.
You have to think about what the post is going to achieve. Is it traffic? Is it leads? How much should I expect? You have to think about how much time is it actually gonna take. Am I gonna be spending 10 hours on this post, and will it be only generating 10 percent more leads, or could I be spending an hour on a post and get 20 percent more leads? We also have to keep our audience squarely in mind and especially what they might think over time, so is this helpful for us building our business in the long term? Will this set us up a year or two years from now to be in a really great place as a business? So those are the things I kept in mind. Obviously some are very, very zoomed in, and some are very zoomed out, but that’s really just what I kept in mind as I was designing that.
Dan: So could we expect a new office kitchen vertical on the HubSpot blog?
Ginny: We’ll see, we’ll see. It might be a new experiment.
Dan: Awesome. Now I’m just giving you a hard time. I found this experiment and this post super valuable, and it made me rethink our own editorial strategy, so thank you so much for sharing.
Ginny: Well, thank you for reading it. I know it was a very, very long post, so it’s nice to have someone to geek out over this stuff with.
Dan: Totally. Thank you so much for taking the time to chat, Ginny.
Ginny: It was absolutely my pleasure. Thank you.
Stephanie: That was Ginny Soskey, Marketing Blog Editor at HubSpot. Have you tried any publishing experiments recently that you think we should know about? If so, e-mail us at podcast.unbounce.com. We’d love to hear about what you’re testing. That’s your Call to Action. Thanks for listening.
Transcript by GMR Transcription.