Marvel’s Digital Comic Vault Experiment: Confirmation Bias Afoot?

This week’s column is going to be short and to the point, because sometimes a lot of exposition (aka The Claremont Effect) can cloud the point.

David Brothers pens a fantastic column investigating Marvel’s decision to implement the “Vault” concept for a few of its digital comics. For those who aren’t familiar with Disney’s Vault Concept, David describes it aptly:
Movie vaults are a concept spearheaded by Disney with the Disney Vault. Essentially, Disney will release a movie for a limited amount of time and then take it off the market for several years, after which it will be re-released to great fanfare. Disney says they do this to keep the movies fresh for new generations of children, but there are children born every minute, so there will always be a fresh audience for Disney movies, whether or not the films are available at Best Buy or not.
David goes on to eloquently explain why a vault, for digital comics, fails on many levels. And I agree. Please do yourselves a favor and read David’s column intently if you haven’t already done so.
Marvel Vault Picture from Marketing PresserAs I read it, I was struck by the fact Marvel is testing this concept out with just two comics: Daredevil #1 and Invaders #1. At first glance, one might think this makes all the sense in the world. Choose a few comics out of thousands in the catalog and see if the threat of putting them in the vault helps drive increased sales.
Here’s my problem…I fear this is going to invite confirmation bias. Confirmation bias, in its simplest form, is the idea that we all look for data points to confirm EXISTING beliefs.  We’re all guilty of confirmation bias in some part of our lives. But just because confirmation bias exists, doesn’t make it acceptable, particularly when it comes to the world of business. As an investor, I see confirmation bias all the time by investors, media covering the industry, or executives running their companies. It’s very easy to believe something and then focus on finding things that support your view. But the best kind of due diligence does just the opposite.
I BELIEVE X and look for data that supports X while ignoring anything that supports alternatives Y,Z
Proper Due Diligence
We examine the data points for X,Y & Z and come to a conclusion based on what the evidence tells us
Getting back to Marvel and the Vault…I wonder how choosing two comics, particularly legacy issues that I suspect are not in high demand, will inform Marvel's marketing executives on whether to expand the idea of the vault. Let’s say, for example, Daredevil #1 was purchased 50 times, and the announcement of the Vault raises sales to 100 downloads. A marketing executive wanting to prove that the Vault is a good idea might look at that data and say, “See! Daredevil #1 download sales doubled after we announced it was going into the vault.” But here’s the problem…absent a broad range of different comics, the sample size could lead to a false conclusion. Because if the average Marvel comic is actually downloaded 1,000 times, would the vault double those sales to 2,000? Or perhaps would it raise sales to just 1,050 (the same net new adds of +50, but as a percentage much less significant).
We just don’t know, and as an analyst my hackles are raised when I see such a decidedly small and focused subset of comics that are going to serve as the “test case” for this vault concept. If Marvel touts the “success” of their vault in the coming months, I sincerely hope they have the intellectual rigor to then expand the testing to a much broader grouping of comics rather than simply convince themselves it’s a great idea and go full bore.


Jason is a mutant with the ability to squeeze 36 hours into every 24-hour day, which is why he was able to convince his wife he had time to join the iFanboy team on top of running his business, raising his three sons, and most importantly, co-hosting the 11 O'Clock Comics podcast with his buddies Vince B, Chris Neseman and David Price. If you are one of the twelve people on Earth who want to read about comics, the stock market and football in rapid fire succession, you can follow him on Twitter.


  1. I’m in agreement that the vault idea is crap – but when they talk about removing it from ‘digital services’ does this also include their subscription service which does not operate on a per comic price, but instead a flat monthly rate?

  2. And Once again Mr. WOOD hits the nail on the head….

  3. On the one hand, the Disney Vault is a pretty savvy marketing idea. On the other, it’s a really dumb ploy that I can’t believe people fall for. I’ve never been motivated to buy a Disney film before it’s been “vaulted” or after it has been “broken out” of the vault. Indeed, I’d say I’ve resisted buying any Disney DVDs for that reason. This seems like a really bad test for this type product. We’re not vaulting Spider-Man #1/Amazing Fantasy #15 or X-Men #1. We’re vaulting things that, let’s face it, people aren’t pounding down doors to get. The other odd thing about this is that there are only a few non-digital options people have for these two specific titles. Very odd indeed. Digital Comics seems to take two steps back for every step forward (“Disappearing” books like Ult. Thor #2 and now books that might not be there next time you log in, lest they be placed into a non-existent vault.

  4. There is one thing everyone is overlooking… How long these things actually stay “in the vault”.  Don’t you guys think it is way too early to say this is a good or bad business plan.

    the Tiki 

  5. Economists (and many other social scientists) are constantly in search of a counter-factual.  For example, if we raise taxes and see that the economy improves we might, foolishly, conclude that increasing taxes stimulates the economy.  The problem is that we don’t know what would’ve happened without the tax.  It is entirely possible (and highly likely) that something else stimulated the economy.  As a result of this issue, economists utilize interesting (to me) econometric techniques and look for “natural experiments” that allow us to construct a proxy for the true counter-factual.  If you don’t know (approximately) what would’ve happened, then the confirmation bias is a MASSIVE problem.

    In the case of this vault model, a valid counter-factual must be created in order to determine if it worked well.  I’m assuming Disney has done this research and discovered that the vault approach is more profitable (the only true motivation).  It sounds like Marvel is running this little experiment to test the waters.  I sincerely hope they fully understand how to determine if it was the right approach.  (I assume they have a collection of comics that are at a similar popularity/sales level that aren’t being vaulted that they will attempt to use as their counter-factual.)

  6. @thefreakytiki  If one of the main aims of digital comics is to increase access beyond the tiny comic book audience then restricting access — in an attempt to artificially create demand — is certainly a questionable business practice.

  7. @Conor is right, as our digital IP economy was pretty much created solely to disperse creative content around the web.

  8. @conor  They see it differently.  

    Are sweets as special to a grown up compared to a child?  No, because you can have them anytime you want.

    People are mistaken by putting all their hopes on digital as the savior.  Digital comics are surrounded in a sea of Angry Birds, NPR podcasts and prose digital books.  These stories are already available in umpteen different formats.  The “build it and they will come” attitude many have is a waste of time.  Digital isn’t the end all be all… advertising the damn product is.  They need a street level awareness/push.

    When I’m sitting on the train reading a monthly comic, others see the comic and sometimes say something.  When I read a digital comic on my iPad on the exact same train, people see Apple hardware… not comics.

    Know what I mean?  😉

    the Tiki 

  9. I don’t think it’s going away from Marvel’s Digital Subscriptions.  (Which is a much better deal, as I’m sure anyone would tell you once “Crunching the numbers”)

    Daredevil #1 is an excellent origin issue.

  10. the way taking a physical piece of media off the market, and re-releasing it works best, is when the re-release has some new bonus content or a better presentation.

    re-release a film on blu-ray, and include a few new documentaries of behind the scene info. films like Wizard Of Oz , and any franchise films or tv series usually blow fans away with new added content.

    can digital comics provide the same , upon their “re-release” , and is it content these newbie readers would find inteteresting?

    i’m thinking no.

  11. @Wordbaloon If I’m not mistaken though, Wizard of Oz wasn’t “vaulted” as it were, they just never pressed new DVDs and allowed the current stock to sell out, and then they released the new version with Blu-Ray. 

  12. I’ve never felt the need to read Daredevil #1.  I doubt that if they reissue it in 10 years I’ll feel any different.  I wasn’t even aware that it is easily available anyway.  Its not currently in print…

  13. These products will still be available in other areas which seems to kind of defeat the “Vault” idea. So what if I can’t download it from marvel? I can still buy it in issues (expensive for those older ones), in a Marvel Masterwork or, and this is my main fear, download it illegally. If you try this in the digital age I would think the end result would be a giant middle finger and a turn to torrents. especially since you are already trying to grab a tech savvy consumer with digital comics in the first place.

    Awesomed By Comics supported the downloading of the show Fillmore because disney has not offered it to buy in ANY form. While this is obviously a little different since you can still buy it in a physical media I can still see others using the same rationale.

  14. Crazy. It seems once again the industry shows that it has no understanding of how digital media can help and change it’s business model.  It also doesn’t seem to understand that that they’re competing against the pirates, who are “selling” a product with greater utility and at a lower price, and now, thanks to the “vault” with greater accessibility. If they want to create artificial scarcity with digital media, then first they have to shut down piracy, but I doubt that will happen.  It seems to my untrained eye that the most successful digital outlets have succeeded by making sure that they have as much content available at all times, so that anytime, anywhere, on a whim, you can pick up that song, read that ebook, watch that movie, etc.

    It feels very much like they’re still trying to force the digital world into their physical world business model, and I don’t expect it will work out for them in the long run.  And I agree with Jason that they need to watch how they interpret the numbers.  Even if sales jump before a title goes in the vault, there’s also the statistic of what sales are when it comes out of the vault, and of how many titles they could have been selling all along had the title remained on the market.  And the trickiest number to measure: How many people saw that Marvel didn’t have it, so they looked for an illegal copy?  Or how many never looked at Marvel’s digital distribution because they saw that the pirates already had a bigger selection?  Those are important numbers to consider, but I have no idea how you get at those numbers. 

    I see the digital market place as a way to capitalize on the huge back catalog that exists in the comic industry that, up until now, was difficult monetize because sales had to be high enough to justify printing costs when reprinting. Now, instead of trying to figure out how they can get as much of their catalog on-line as fast as possible, they’re exploring ways put artificial limits on the digital world.  I really want to see the comic industry embrace digital in a way that revitalize the industry, get more comics into people’s hands, and possibly make some fringe titles profitable enough to keep around longer.  Instead it seems that all they want to do is shoot themselves in the foot.

  15. The problem with this is they aren’t removing those comics from the marketplace. They’re removing a single format of the issue. What about all the singles and collected editions? This won’t do anything.

  16. @PraxJarvin  you kind of missed my main point, which is… when something gets re-relased it usually has more than the original material , making the new purchase more inticing.

  17. Ah the growing pains of digital comics…how they ache!

  18. But whether a vault announcement increases downloads by double, or just by 50, that would still be an increase.

    I don’t like the idea, but I would definitely trust Disney to know the pro’s and con’s of vault ideas since they invented them. If the vault idea doesn’t work for certain titles, and it would pay off in the long run for the titles to simply be available all the time, then I’m sure they’ll go back to that. They’ll do whatever will yield the most profit. And they’ll totally ignore what many fanboy pundits have to say. Evidence: They’ve been able to introduce digital comics at higher price point than most of us were recommending.

    We can not like this but it doesn’t really matter. Disney will manage this for their best interests. We can act like each of us personally knows stunningly obvious ways to sort everything out, increase comic sales to the billions, all for $0.01 an issue. But we really don’t have any idea of that. The main thing I’ve gotten from so many articles like this one–sorry!–is that no one fanboy has a wide enough grasp of the marketplace or how it would operate. Sure, some of us are self-proclaimed experts at tiny aspects of the puzzle, but you can be an expert at analysis all you want…but if you also aren’t privvy to behind-the-scenes economics, then your analysis doesn’t matter.

    Personally I take some resentment at seeing something called The Claremont Effect being posited as some sort of negative. All that says to me is “I can’t deal with a lot of words or information, so I’m just going to be pithy, name-check a few analytical vocab terms, and dash off an article that makes sweeping claims and caters to fanboys’ OWN ‘confirmation bias’ about Disney/Marvel being mean.”

    Just because we don’t like it doesn’t mean that “vaulting” isn’t a legit way for Disney/Marvel to make the digital model work. Because, let’s face it, despite all of the optimistic talk I’ve heard for years and years, digital comics have not revitalized the industry in any way.

  19. @froggulper  Yes, why hasn’t this thing that basically just started happening in any meaningful way in the last six-eight months revitalized the industry?

  20. Marvel has been playing around with digital comics since 1996 with Marvel Digitial comics that later became DOT comcis and so on and so on/

    They should be the subject matter experts at this point wil all the data they have collected and best practices they should have developed by now but instead they seem more lost than ever on what to do.

    I think they need to sack up and lead- take a risk- lose profit on a bunch of xboxes initially only to become the leader  in the industry and reap the profits later on -Something!

    But for them to still be wading in this pool after being an early pioneer is embarrasing- Speaking from a business stand point.

  21. @conor  The comic book industry is a questionable business practice. This idea is dumb.

  22. Dumb, dumb, dumb idea – surely part of the point of digital comics is so that you offer a product that is never at the risk of going out of print.  Restricting access is the same a limited print run, essentially punishing people who don’t get there quick enough.

  23. another aspect of vaulting movies is that it creates a collectible/investment component due to the fact of its limited edition and iconic status. It also increases the need to buy NOW because you won’t be able to get it at retail price for very long before the secondary market drives prices too high and you’d have to wait years to get it again. 

    What collectibility does a digital comic have? Seems like a lot of the components of what makes vaulting successful is lost on a downloadable product. 

  24. Ass-backwards.

    No consumer gets excited about media being PLACED INTO the vault, they get excited when it re-emerges!

    We are still in the very early phases of their digital roll-out so if any excitement is going to be generated by Invaders #1 in this day and age, it has already happened. They’ll get far more advertising mileage by pimping the first releases of genuinely pivotal stuff like Lee/Kirby FF than they will from this. Go to the NYT and give them a ready-made article about the rebirth of legendary comics in digital form and they’ll give you free column space, guaranteed. No-one will give a crap about this move, though.

  25. All I see this practice doing is creating a spike in piracy on these issues after they’re placed in the Vault. Society is moving towards on-demand entertainment in all areas. This is a regressive step.

  26. This concept is a bad business decision. They are trying to create artificial scarcity in the digital realm. It’s not going to work. People can already get any comic they want for free if they know where to look, so pulling legitimate copies from sale will only make piracy worse in the long run.  Marvel should be doing quite the opposite, which is making day and date releases available on the Comixology app, and they should take their Digital Comics Unlimited and make an app for that as well, so that my $60 per year will allow me to read the DCU stuff on my iPad, where it would make more sense, as I hate reading them on the PC.

    Marvel is showing how they are still stuck in yesteryear’s thinking.  

  27. @Cormac  That’s a good point.  Also if you consider the print and digital to be the same item, then it already has come out of the vault.  Basically Marvel (and DC, and others) has been sitting on a huge vault of content for ages, and internet has finally created a way for them to get it back out there, and they immediately start figuring out how to lock it back up.

    Also, Disney’s concept, if I understand it, is that these movies are classics, movies that everyone has seen at some point, but maybe didn’t own.  Or maybe didn’t own in the current format. Or maybe they want to pass them on to someone else.  I don’t think my 7 year old nephew is on pins and needles waiting to see the Jungle Book.  It’s not even on his radar, bombarded as he is with new content, but I might buy it for him because I love it.  The great thing about bringing these old comics back out is that most people haven’t read them.  I keep hearing that problem with the comics industry is that it’s not attracting new consumers.  The vault concept seems to be predicated on repeat consumers, but first you need new consumers before you can get them to be repeat customers.  Having old runs available digitally is great for someone starting out who wants to catch up on a character’s history.  This website is great about talking about “jumping on points.”  The problem with jumping on points is that they’re moving targets.  You see the Iron Man movie, decide you want to check out the comic book, and find you’re in the middle of a story that you have no context for.  But with digital publishing, everything could potentially be in print, and anyone could jump on at any point, even at the very beginning.  Unless you decide to time-release your old stuff, in which case the potential new customer may well have forgotten about it.  For my part, when I’m in search of a piece of music or video, I look for where I can buy it, and if it’s not there, I either look to see if I can find it for free, or I forget about it.  Very very few things actually get noted and looked for again.

    But, yeah, I don’t know marketing.  I don’t have a business degree or any inside information on the comic industry.  But I keep hearing about how the industry needs new consumers, and I keep seeing the industry act like their only customers are the long-term, die-hard fans who aren’t going to notice that the industry is feeding them digital media with the same restrictions as if it were print.  So it is without any expertise.  That’s true.  But if I were dipping my toe into the waters of comics and thought Daredevil was an appealing character, it’d be a shame if Marvel was keeping me from reading the back issues because they wanted to drive up the value to suck more money out of their long-term supporters.  If I can’t read back issues of Daredevil to satisfy my curiosity, I bet I could get that old Hulk movie with Daredevil in it from Netflix.  Which one is going to make me want to come back for more?