How to Beat Google at Search for $50,000 in 10 Easy Steps

sumo_mismatchDon’t you hate blog posts that have some stupidly audacious title like “How to Beat Google at Search for $50,000 in 10 Easy Steps” and then fail to deliver the goods? Well, get ready, because this isn’t going to be one of those posts.  SetJam is the world’s best search engine for online TV shows and movies.  We really did beat Google (and every other company that’s built a similar search engine).  We really did it for $50,000 ($50,231 to be exact).  And yes, I’m actually going to explain to you how we did it.  And when I mean “explain” I don’t mean some vague platitudes about hard work and listening to your users.  I’m actually going to give you “secret” information–the real details that let us pull this off–details so good you could copy us.

One word on why I’m doing this.  Partly, it’s my belief in openness and sharing.  I think we’re all in this “thing” together–call it technology, entrepreneurship, life, whatever.  I owe a great deal to this most open of platforms, the internet, and if I can help you make it better, the better for all of us.  I’m also doing this out of self-interest though.  I don’t have the marketing power of Google (or any of our competitors).  By giving up the goods, I expect you to:

  1. Use SetJam.
  2. Tell your friends, your family, the guy you buy coffee from–EVERYONE to use it too.
  3. Tell us how to make it better.  We know it’s broken, but we’re too close to it to see HOW it’s broken.  We need you to take the time to let us know.

Okay.  To begin with I’m going to start with 3 of those platitudinous rules, because they apply to all startups.  After that, I promise to get into the secret details (with video examples!) that are specific to how we built a better search engine than Google.

Step 1: Don’t try to beat Google. Try to solve a problem.  We weren’t TRYING to build a search engine.  We were just trying to watch TV shows and movies online.  If we would have started out with the goal “I’m going to beat Google at search”, I think we would have failed.  That’s because we would have made our search TOO much like Google.  We would have seen all the amazing details they do that make web search great and we would have ineluctably copied them.  Since we were focused on solving a real world problem though, we were able to take a completely different approach.

Step 2: Don’t be afraid. I would have never tried to build a search engine because search engines are really “hard-core” technology that operate through “secret algorithms” and cost billions of dollars to make.  Not true.  Search is just like any other technology problem.  The funniest thing to me is that I was so terrified of search, that I didn’t even realize what we’d built until I tried to explain it to other people.  I would use phrases like “TV guide for the web”, “A concierge for online video”, “Your DVR for online TV”.  No one had a clue what I was talking about.  I’d then try to explain it with “Just add your favorite shows, and we’ll bring you the best links, perfectly organized”  I kept thinking, if only there were some easy way to explain a magic text box that brings you the best links.  I kid you not.

Step 3: Get your product out there and listen. We put SetJam out into the wild well before it was usable.  We did this because I really wanted to see our mistakes early.  My initial vision for SetJam was a way to build a TV queue (much like you do with a DVR) and then to bring you the best links to watch your shows.  The problem with this is that to build a queue, we need to know who you are, which means registering–blech!  To mitigate this issue, we used Facebook Connect to remember who you are instead of making you create another account.  Instead of mitigating the issue, however, it exacerbated it.  Not only did we get complaints that we were forcing people to authenticate before seeing any results, we were forcing them to authenticate through Facebook so we could spam their friends!

These problems seem obvious now, but when you’re in the heat of the product battle, you miss obvious things.  We KNEW our results were amazing, so we never thought that anyone would need PROOF before starting to build a queue.  Why not save the step of having to manually add the search results?  We KNEW we were using Facebook to PROTECT our users from creating another account and we KNEW we would never write anything to their News Feeds unless they told us to.  Our users had been burned before though, and they didn’t trust us yet.  If we hadn’t gotten SetJam out and listened, we’d probably lose most of our visitors BEFORE they could even see what we’d built.

That’s it for the platitudes.  Now let’s get into the secret details.  For most of these, I’m switching over to video mode, so I can show you what I’m talking about.

Step 4: Focus your corpus. Oh shit… the secrets are in Latin!  Don’t worry, this is just search engine jargon.  Your search “corpus” is the body of information you’re trying to search against.  If you’re a small team with limited resources, I do not think you’re going to beat Google at searching the entire internet.  By focusing on a smaller (yet still REALLY important body of information, like TV shows and movies), you can do things they can’t though.  Check out the video:

YouTube Preview Image

Step 5: Index what Google doesn’t. Google doesn’t index all the data in the world.  In fact this may surprise you, but Google doesn’t even index the MAJORITY of the data in the world.  That’s because MOST of the data in the world is locked up behind password protected sites or not even networked.  Bring that information to your search and you’ve gone a LONG way to beating Google.  Check out the video:

YouTube Preview Image

Step 6: Cut out the clutter. One of the biggest problems we face as humans today is that there is just TOO much information.  This means that getting the MOST information (by indexing things Google doesn’t) is just the beginning.  Your next step is to get rid of the stuff that doesn’t matter.  Check out the video:

YouTube Preview Image

Step 7: Organize your results for a purpose. Google has no specific purpose.  Because of this, it organizes its search results in a very generic way.  If you’ve followed step 1 and started by trying to solve a real problem, you can organize your results in a way that serves that SPECIFIC purpose.  Check out the video:

YouTube Preview Image

Step 8: Prove the negative. Okay.  I can’t BELIEVE I’m giving this one up because this is some secret mojo that makes SetJam great.  Basically this rule means that sometimes it’s just as valuable to show what is NOT available as it is to show what IS available.  You all OWE me for this one.  Check out the video:

YouTube Preview Image

Step 9: Keep it simple. This rule is straight out of Google’s handbook, but it bears repeating because if you do NOT follow it, Google will kick your ass.  I know you want cool feature X,Y,Z, but let me just tell you that your users will not get it.  Never underestimate how complicated and confusing new software is to your average person.  If you can do one thing and do it in a way that people actually get, you are WAY ahead of 99% of your competition.  Check out the video:

YouTube Preview Image
(Btw, if you’d like to see how I reacted when I found out that Clicker, an $8 million dollar startup, launched 3 weeks before us, check out this post.)

Step 10: Test Driven Development. This last rule is not one that I can really explain in a video, but it is absolutely critical if you want to beat Google at search.  Here’s a scenario you’ve probably never considered unless you’ve tried to build a search engine.  You’ve just finished your prototype and you’re looking at your results.  You type in a result that gives you bad information.  You figure out the problem and change your code to fix that problem.  How do you know that you didn’t just ruin the results for the 9 million OTHER things you index?  Ah-ha… you don’t unless you’ve been building unit tests and data integrity tests all along the way.  Without these tools you will be in a never-ending cycle of fixing problems while simultaneously causing them elsewhere.  If you haven’t been a believer in test driven development before, you better GET that religion before you try to become a search company.

That’s it.  10 steps to do the impossible.  Now get out there and start making the world better.  Before you do though, signup for our private preview at www.setjam.com.  To be fair SetJam still has a LOT of issues.  We’re still battling dupes, merged series, and missing links.  We still don’t index half of what we want to.  Despite all this, SetJam is still the easiest way to find and watch full-length TV shows and movies online, and with your help, it will only get better!

  • http://eastriverbaby.blogspot.com/ Jess

    I want in.

  • http://bit.ly/ARTcd Shakir Razak

    Hi,

    Nice post. ;)

    Next step, become a verb!

    Kind regards,

    Shakir Razak

    • Marcos

      Ha, maybe “I'm going to 'jam it tonight”.

      There's more humor in that than might be known to many. There's an old story of how Steve Wozniak (Apple computer founder-need I say?) who, back in the day, besides making Blue Box's and other phreaking equipment made a handheld VHF TV transmitter which would jam the frequencies of various television channels. He would cruise around his Cali 'burb on SuperBowl night and just when there's was about to be a big play, flip on the transmitter and jam the station, watching all the brutes stomp up and down via their living room window, frantically re-arranging their antennas trying to see what happened (pure analog, no DTV replay here!).

      Ha! Nerds be 'venged. (Damn, what's the word I'm looking for?)

      Mark

      Btw, Ryan, nice work man! I'm impressed and encouraged.

  • http://filmaster.com michuk

    Good post, although there are many sites that beat Google at search in specific areas. One site that somes to mind first: PirateBay.org

    • http://www.setjam.com drstarcat

      Amen to that. People don't realize how good the bay is as a search engine.

    • http://www.setjam.com drstarcat

      Friends, Family, and Once and Future Colleagues,

      I’m writing to let you know that today SetJam was acquired by
      Motorola Mobility. We are all very excited about this transition
      here at SetJam. Motorola and SetJam share the vision of making content delivery, discovery, and consumption seamless across any screen, and as a world leader in video technology, Motorola will provide us with unprecedented levels of reach and distribution.

      I want to take this moment to personally thank you for your continued encouragement and support over the past two years. None of this success would have been possible without your backing and belief in us. While this acquisition was in process, we’ve had to be far less communicative than we normally would be. We greatly look forward to reconnecting with all of you in the New Year. In the mean time, we wish you a happy and healthy holiday season!

      Cordially,

      Ryan

      Ryan Janssen
      (Former) CEO, SetJam

      ______________________________________________________________________
      If you no longer wish to receive these emails, please reply to this
      message with “Unsubscribe” in the subject line or simply click on the
      following link:

      http://cts.vresp.com/u?366f85468a/081e938a5d/mlpftw

      ______________________________________________________________________
      This message was sent by Ryan Janssen using VerticalResponse

      SetJam
      154 Grand Street
      New York, New York 10013
      US

      Read the VerticalResponse marketing policy:
      http://www.verticalresponse.com/content/pm_policy.html

  • http://twitter.com/Readmore Readmore

    Great post! I just got into the beta today and I'm excited to try out Setjam.

    I'd like to know if you have any plans for an API? I understand there are obviously business reasons why that may not be a good idea but accessing your show information could really help out an indie developer like myself.

    • http://www.setjam.com drstarcat

      We've definitely got plans for an API. Our goal is to help users to share
      their movie and TV preferences with any site. You'll be hearing more about
      it early next year.

    • http://www.setjam.com drstarcat

      Friends, Family, and Once and Future Colleagues,

      I’m writing to let you know that today SetJam was acquired by
      Motorola Mobility. We are all very excited about this transition
      here at SetJam. Motorola and SetJam share the vision of making content delivery, discovery, and consumption seamless across any screen, and as a world leader in video technology, Motorola will provide us with unprecedented levels of reach and distribution.

      I want to take this moment to personally thank you for your continued encouragement and support over the past two years. None of this success would have been possible without your backing and belief in us. While this acquisition was in process, we’ve had to be far less communicative than we normally would be. We greatly look forward to reconnecting with all of you in the New Year. In the mean time, we wish you a happy and healthy holiday season!

      Cordially,

      Ryan

      Ryan Janssen
      (Former) CEO, SetJam

      ______________________________________________________________________
      If you no longer wish to receive these emails, please reply to this
      message with “Unsubscribe” in the subject line or simply click on the
      following link:

      http://cts.vresp.com/u?366f85468a/a0535a421d/mlpftw

      ______________________________________________________________________
      This message was sent by Ryan Janssen using VerticalResponse

      SetJam
      154 Grand Street
      New York, New York 10013
      US

      Read the VerticalResponse marketing policy:
      http://www.verticalresponse.com/content/pm_policy.html

  • Pingback: Borys Musielak (michuk) 's status on Tuesday, 10-Nov-09 14:26:48 UTC - Identi.ca

  • http://reecepacheco.com reecepacheco

    Ryan,

    Great post. Love the transparency. I have a much better grasp of SetJam now (than I did after NYTech Meetup).

    Keep up the great work.

  • http://m.hapnin.com theschnaz

    Interesting post, I like how simple you kept everything.

    If I search for a show and nothing is free online, does telling me it's available on Netflix help? If I had a Netflix account, I'd be there. If I don't, there's probably a reason.

    I think it's smart to index things Google doesn't. What is your opinion on less than ideal video hosts? Flickpeek.com is a great place to find videos, they have HBO shows too. http://www.flickpeek.com/tv-shows/True-Blood/ Would you index that site? If so, how would you add value that they don't?

    • http://www.setjam.com drstarcat

      Good questions. I think showing people how much is available instantly on
      Netflix is a good thing, even if they don't have an account. We hope that
      by showing them how good their site is, they might reconsider and join.

      As to illegal content, we're filtering that out for a number of reasons.
      First, we want to develop a community that can be a voice back to the
      content companies, so we can change some of the things that are wrong about
      distribution. We lose that voice once we index illegal stuff. Second, love
      good TV and we know that if it all goes illegal, companies won't be able to
      invest to create it. We don't want a world where the only thing that exists
      is user-generated content. Finally, we want SetJam to be a place where
      people can store their “TV Identities” for the long-term, so they have
      ratings and favorites that will work anywhere. We can't be a safe place for
      people if we're doing things that are illegal.

      That's where we're at now. Who knows how this is all going to change over
      the next couple of years though.

      • http://m.hapnin.com theschnaz

        Cool. That makes a lot of sense.

        From reading/watching your post it seems that you have the “flow”
        down. People come to SetJam, search for a show, find the show easily,
        watch show, repeat. All the extra fluff, the clips, bios, and long
        descriptions aren't needed.

        I'd love a system that tells me when my new shows are available.
        Hulu's queue is a great idea, but the implementation is terrible (or
        I'm not smart enough to use it.) I like 10 shows, just tell me when
        new ones are available! This could be part of the “TV Identities” you
        mentioned.

        Good luck! If you ever need testers or feedback, let me know.

        Greg

  • Marcos

    Ha, maybe “I'm going to 'jam it tonight”.

    There's more humor in that than might be known to many. There's an old story of how Steve Wozniak (Apple computer founder-need I say?) who, back in the day, besides making Blue Box's and other phreaking equipment made a handheld VHF TV transmitter which would jam the frequencies of various television channels. He would cruise around his Cali 'burb on SuperBowl night and just when there's was about to be a big play, flip on the transmitter and jam the station, watching all the brutes stomp up and down via their living room window, frantically re-arranging their antennas trying to see what happened (pure analog, no DTV replay here!).

    Ha! Nerds be 'venged. (Damn, what's the word I'm looking for?)

    Mark

    Btw, Ryan, nice work man! I'm impressed and encouraged.

  • http://watch-free-movie-online.net alex777

    watch free movie

  • air max shoes

    I totally agree the standpoint of upstairs, and I believe this will be a trend. I often come this forum , rom here I learn much and know the newest tide! the content here constantly update and I love it! Another I know some websites which often update their contents, you guys should browse if into321 you are free.

  • http://www.usa-basketball-shoes.com basketball shoes

    Here elaborates the cake-like.com matter not only extensively but also detailly .I support the write's cake-like.com unique point.It is useful and benefit to your daily life.You can cake-like.com go those sits to know more relate things.They are strongly recommended by friends.Personally

  • Kchase77

    I'd like to find someone to partner with for a Product Search engine. Write me at kchase77@gmail.com if you are a technical wizard.

  • Webbhotell

    well I have other approach that can beet Google user base in rapid time. But not the programing knowledge…

  • http://www.onchannel.net/ Amalia Dragutzza

    another great movie sharing webite is http://www.onchannel.net because has movies and tv shows posted everyday.

  • Pingback: Online Marketing Skills