TheFunded's Ominous Algorithm

TheFunded, a site where VC's are rated by entrepreneurs, has been around for a while and gets a lot of press in the blogosphere (it sure does piss some people off). I originally greeted the idea and supported the concept strongly. I believe it's a good thing to have transparency in this market and I looked forward to both good and bad ratings. It's good to know what people like about you and even more so what they dislike about you. As a VC or a fund you can learn from this. Feedback is always good. In general, I believe people know how to read these types of rating sites and always discount the worst and best of the ratings (hence I never really cared or worried much about potential vengeance motivated ratings). Nevertheless, after being at the number one spot on the list for one day, we were all of a sudden flagged "due to suspicious activity". You can imagine this was surprising. I had some dialog with Adeo Ressi, the founder and tried to resolve this issue. Originally, the response was the aforementioned "suspicious activity". Very recently though it became "the Algorithm". This algorithm is all-knowing now and drives what happens on the site. This article (sent to me by Ressi) adds some more flavor to this.  What's my problem now you ask? Well, we didn't try to game the system. I actually also highly doubt Fred Wilson, as mentioned in the linked article was trying to game the system. Actually, we tried to support it. We pointed entrepreneurs to the site and recommended that if they had feedback, they should give it there. Well, now of the 11 or so ratings we have, 8 I believe are considered  "suspicious" by the algorithm and have been discounted by the site (I do give the site credit though for leaving them up there). I apologize to those entrepreneurs who took the time to provide this feedback to only have it devalued on the site by "the Algorithm". I understand that measures to avoid the system being "gamed" need to be in place but somehow everything about TheFunded seems tainted due to this algorithm. The ratings seem tainted, the scores seem tainted, and even the fact that VC's invested in this makes it seem tainted. How are the VC's who invested in the company or on the advisory committee (there is supposedly one) rated on the system I wonder? I still have hope for the site though as something like this thrives on growth and a large number of reviews where you can filter out the fluff. If it continues to grow it could evolve into an interesting tool for both entrepreneurs and VC's. It unfortunately has a ways to go in my opinion and "the Algorithm" needs significant tweaking before there is true credibility for TheFunded.

Comments

[Founding Member]

The algorithm was designed to spot any unusual activity in the numerical rating ("rating") and written review ("review") systems. Approximately 15% of all firms on TheFunded.com have unique characteristics that make them different from hundreds of other funds rated and reviewed by thousands of CEOs. These firms with "suspicious" content included both top rated and lower rated firms.

What are some examples of suspicious content? Imagine if over 75% of a firm's rating and review content was supplied by CEOs that just logged on once to write something about only that firm. Imagine if a large portion of review contained superlatives, like "best," "greatest," "worst," "horrible," and exclamation points. Imagine if numerous CEOs that reviewed the firm clicked a checkbox stating that the firm asked them to write the review in question.

TheFunded.com created a scoring system to evaluate a series the level of suspicious activity, and, if a firm scored beyond a certain threshold, the firm was flagged with a warning message for approximately 45 days. After the warning period, no content was deleted, but suspicious numerical ratings were eliminated from the fund average rating. The net result is a more accurate system.

TheFunded.com rating leaderboard was previously inaccurate according to the entrepreneurs that used the free service, which hurt CEOs looking for accurate information. For example, small funds with a limited operating history and no track record were listed as the best funds in the world. Fred Wilson publicly stated that he did not deserve his place on the leaderboard. The corrections implemented by use of the algorithm have made the leaderboard much more accurate to the opinions of the broader Membership.

[Paul J.]

Adeo, thanks for trying to clarify. Like I said, this is a nice project trying to become a tool and I hope it's successful. Unfortunately, your ratings will nevertheless remain skewed. First of all, you simply don't have enough members reviewing multiple VC's. I don't know what the right number is but in terms of membership it's significantly higher than what you have now to have "truly" representative feedback. Further, most people who review something on a site don't necessarily become serial-reviewers. This is a major problem with all sites on the Web where people review things. The majority of the reviews will initially be written by people either giving really good feedback or really poor feedback. Everything on TheFunded has to be taken with a grain of salt (and this is not your fault...it's simply the way things go). Finally, you penalize smaller funds and less experienced entrepreneurs (this is where I believe the algorithm needs the most tweaking). Most entrepreneurs can't go on TheFunded and review 10 VC's. They've maybe met two or three in their whole career thus far and really don't feel compelled to review more than one. Additionally, smaller funds with less partners simply aren't seeing as many companies and engaging with that many entrepreneurs. Hence, their reviewing pool is much smaller. Your point about smaller funds not belonging at the top of the leaderboard is completely negated in this regard since there may be funds with less track record or operating history who are liked and respected by entrepreneurs. As I already said, I am in favor of TheFunded succeeding and wish you all the best with this daunting project. It's clear the site is evolving and getting better. Hopefully we'll see the day when there's enough volume for it to be a win-win situation for both entrepreneur and VC.