Stanford's Bold Experiment: Free Public Courses in AI, Machine Learning, and Databases

Inspired by Salman Khan of Khan Academy, Stanford computer science professors will make three courses available to the public this coming fall: artificial intelligence, machine learning, and databases. Although there have been previous efforts by universities to make course materials publicly available (e.g., MIT OpenCourseWare), Stanford's latest effort is unique in allowing students from around the world to participate in the course, complete with homework and exams. Upon successful completion of the course, students will receive a statement of accomplishment.

Initial interest in these public courses has exploded: Over 60,000 people have signed up for the artificial intelligence course, taught by Peter Norvig and Sebastian Thrun. With so many excited (and computer-literate) students from around the world participating in the same courses, some of these students may decide to create new educational technologies for student discussions and collaboration. In addition, these students' online interactions may indirectly result in a collective knowledge resource that trumps any public sources currently available on these subjects.

It is an exciting time in online education.

Links:

A Musing on Game Theory Dug Up from the Past (8/18/05)

what is the mathematical basis for the concept of strategy? a perfect strategy in a game with complete information is one that guarantees a win. but we as humans talk of good strategies and bad strategies all the time without any clue as to what the perfect strategy for the particular game looks like. for example, in the game of go, there are moves that are considered good and moves that are considered bad, but from a game theoretic standpoint, there are only perfect moves (those that preserve a game winning strategy for the player) and vulnerable moves (those that allow the second player to seize the winning strategy).

strategy, in essence, is a set of general guidelines that give the player a better chance of winning the game. the point of delving into the very basis of strategy is that computers have no intuition and thus, to program a computer well, it would seem helpful to define the concept of strategy in terms that computers can understand.

in the game of chess, deep blue's defeat of garry kasparov in 1997, to many, signaled the changing of the guard from man to machine. but in games such as poker and go, the inherent computational complexity of the game makes it difficult for computers to approach the level of top humans. in order to tackle games that are computationally complex, it seems to be necessary to give the computer tools to simplify the game, in essence, create human-like strategies that guide the computer rather than have the computer calculate possibilities in a great number of directions and then defining some metric of desirable positions. this is the problem we attempt to tackle.  

---

in game theory, the way to determine the perfect strategy is to calculate all possible moves by both players and then to choose the decision tree that results in a win. however, if this were the only way to approach games, we would have no insight into any game whose possibilities were too large to compute. obviously, this is not the way humans approach games. there are human experts in games such as go, poker, and chess -- games that have not been solved in a game theoretic sense -- and to guide themselves during play, these players rely on strategic principles they have developed through experience and thought. but from a game theoretic standpoint, what exactly is a strategic principle? if for example, there is only one perfect strategy that can be calculated through game theory, then what do we mean by a good strategy or a poor strategy? for any strategy that is not perfect must be poor in some sense.

in all strategic games, experts use intuition, experience, and thought to develop strategies that allow them to consistently beat poorer players. those players who play multiple games might be an expert in one game, but their lack of experience in another game may cause them to be less than skillful in that one. but shouldn't there be some general principles that the expert of one game can carry to another? or put differently, shouldn't the training and effort one put in a strategic game pay off indirectly as that player tries to learn another game?

perhaps the lack of literature on general strategy is due to its former lack of applicability. a player who wanted to learn poker would read a poker book; a player learning chess would read chess. there was no real need to generalize concepts such as strategy for if a player wanted to learn the strategy for a game, it would be much more practical to learn that specific strategy as opposed to relying on generalities.

however, the time has come where computers are very powerful and yet are utterly weak in certain domains. for example in poker, an expert would jump at the opportunity to play a computer for real money. this is because computers these days that play poker are relying heavily on the strategies that have proven successful in chess--that being calculating a huge tree of possibilities and choosing the best one. but in a game as computationally complex as poker, general strategic principles (in other words, thinking more or less like a human) would seem a logical way to handle the current problem of technological limits.

just as many business executives have been quite successful without learning any formal economics, many thinkplayers have been successful without any notion of what game theory really is. however, with the demands of artificial intelligence, the time has come to develop a theory of general strategies for games, so that instead of thinking up a billion random thoughts and pruning down these thoughts to one, a computer can think of a much smaller number of sensible thoughts, and from there, make a decision that is in some sense, intelligent.  

Paul Graham on Generating Startup Ideas

Across his essays, Paul Graham gives a step-by-step process for generating startup ideas:

1. Pick a project.

Put in time how and on what? Just pick a project that seems interesting: to master some chunk of material, or to make something, or to answer some question. Choose a project that will take less than a month, and make it something you have the means to finish.

"I need help picking an idea." Response:

So if you want to start a startup and don't know yet what you're going to do, I'd encourage you to focus initially on organic ideas. What's missing or broken in your daily life? Sometimes if you just ask that question you'll get immediate answers. It must have seemed obviously broken to Bill Gates that you could only program the Altair in machine language.

"But my project seems trivial and I want to work on something great." Response:

Great questions don't appear suddenly. They gradually congeal in your head. And what makes them congeal is experience. So the way to find great questions is not to search for them-- not to wander about thinking, what great discovery shall I make? You can't answer that; if you could, you'd have made it.

The way to get a big idea to appear in your head is not to hunt for big ideas, but to put in a lot of time on work that interests you, and in the process keep your mind open enough that a big idea can take roost. Einstein, Ford, and Beckenbauer all used this recipe. They all knew their work like a piano player knows the keys. So when something seemed amiss to them, they had the confidence to notice it.

 

2. Finish the project.

"How do I avoid procrastination?" Response:

Of course, the main reason people find it difficult to work on a particular problem is that they don't enjoy it. When you're young, especially, you often find yourself working on stuff you don't really like-- because it seems impressive, for example, or because you've been assigned to work on it. Most grad students are stuck working on big problems they don't really like, and grad school is thus synonymous with procrastination.

"But I think I like my project." Response:

Another reason people don't work on big projects is, ironically, fear of wasting time. What if they fail? Then all the time they spent on it will be wasted. (In fact it probably won't be, because work on hard projects almost always leads somewhere.)

If you want to work on big things, you seem to have to trick yourself into doing it. You have to work on small things that could grow into big things, or work on successively larger things, or split the moral load with collaborators. It's not a sign of weakness to depend on such tricks. The very best work has been done this way.

3. Repeat the cycle.

Repeat till, like an internal combustion engine, the process becomes self-sustaining, and each project generates the next one. (This could take years.)


Sources: The second quote in Step 1 came from Organic Startup Ideas. Quotes for Steps 1 and 3 came from What You'll Wish You'd Known. Quotes for Step 2 came from Good and Bad Procrastination.

Other related articles by Paul Graham:

Doing a startup while in school: Socially Acceptable Unemployment

Facebook, Google, and Yahoo were all founded by (dropout) students. The press and the blogosphere have focused a lot on the founders' decisions to dropout ("If you want to be a billionaire, don't graduate") and on their ages (reinforcing a youth-centric view of entrepreneurship). But here's a point that hasn't been given enough attention: As students, the founders had the unstructured time necessary to develop wildly popular, free services.

So why isn't it possible for non-students to do the same? It is: See Twitter, Quora, Foursquare. (It gets a little tricky because the idea for Foursquare's precursor, Dodgeball, was hatched while Dennis Crowley was a student.) But note that Twitter, Quora, and Foursquare were all founded by people with the finances and influence that come from past successes (Blogger, Facebook, and Dodgeball, respectively).

Of course, it is possible for first-time entrepreneurs to make great free services (e.g., Blogger, Reddit). But without any short-term income, trying to come up with the next great free software service is a risky strategy.

No one wants to be viewed as a "loser." And being the founder of a startup that hasn't taken off yet sounds a lot like being unemployed. You can get a real job and work on your dream project on the side, but people on ambitious career paths tend to have all of their free time sucked up by their day job.

It's easier as a student, because people are forced to withhold judgment on your career. But once you're out, the clock starts ticking.

PhD vs Startup: Is a Computer Science PhD worth it?

Recently, the Hacker News community debated the value of a PhD degree:

http://news.ycombinator.com/item?id=2019665

The HN discussion was based on an Economist article (http://www.economist.com/node/17723223?story_id=17723223) that made the following points:

  • Academic jobs are scarce. Therefore, many students hopeful of becoming professors will end up disappointed.
  • On average, the PhD has zero (or negative) economic value.

Having spent a few years in a Computer Science PhD program, I'd like to share my thoughts.

  1. It can be misleading to lump all PhDs into a single discussion. Since the software industry is healthy, Computer Science PhDs have a range of career options other than the academic path. Tracy Chou posted a good answer on Quora discussing CS PhDs:

    http://www.quora.com/What-are-the-advantages-and-disadvantages-of-doing-a-CS-PhD

  2. It can also be misleading to ask whether a PhD is "worth it," framing the pursuit of a PhD as an economic decision. A PhD makes the most sense when it opens up career paths better aligned with your natural passions.

  3. A PhD opens doors, but also closes a few. (Some startups have a distaste for PhDs since they expect more money and aren't necessarily better at making software.)

  4. There is a huge opportunity cost (career advancement + money) in spending half of your 20s pursuing a PhD.

  5. I am happy with my decision to pursue a PhD. Over the past few years, I've interacted with motivated peers, got exposed to interesting new ideas, and gained experience coming up with and developing my own projects. I've learned a lot, and I've gained transferable skills. I've also had a lot of fun. But I think enjoying a PhD program requires a certain personality. Do you enjoy thinking deeply about unsolved problems?

  6. You can have a huge impact with or without a PhD. Some technologies required a deep expertise to develop (e.g., VMWare, Google, Akamai). Other hugely influential products did not require as much technical expertise during the initial stages (e.g., Facebook, Twitter). When considering whether to pursue a PhD, ask yourself which career paths better suit your talents and passions. If you still think a PhD is right for you, it might be worth a shot.

Related articles:

Learning to Program (Why Visual C++ is a bad first language)

My high school had a graduation requirement called "Senior Project." During our senior year, we had to devote 20 hours towards learning a new skill. I used this opportunity to learn how to program.

The first choice I made was on which programming language to learn. I scanned the programming book offerings on Amazon to see which languages were popular. I remember seeing C and C++ and thinking, "Those extra pluses must mean C++ is better. I'll learn C++." And then I recall seeing Microsoft Visual C++ and thinking, "This is the language I should learn. Microsoft is legit, and I want to be a real programmer."

So I ended up learning Microsoft Visual C++ using Ivor Horton's 1000+ page book. I learned a little bit, and thankfully, I didn't fail my final project presentation. However, I finished the experience thinking that programming is complex. I was amazed by the number of programmers who must have mastered the 1000+ book by Horton.

It turns out that I picked the wrong first language. Visual C++ is unnecessarily complex as a language to learn programming fundamentals. If I had to pick all over again, I would have picked a language like Python -- consise, intuitive, and readable.

Part of the reason why I chose Visual C++ was because I wanted to build legitimate software, and back then, anything that was "legitimate" had to be a Windows application. However, the essence of programming is better learned without the additional clutter introduced by Windows-specific functionality. Also, if I had a glimpse into the future, I would have realized that in 2010, web applications are perfectly legitimate, and don't require learning any language with "Visual" in its name.

If you want to learn to program, my advice is to pick a language other than Visual C++. Python is a good first choice.

If you are interested in Python, here are a couple of good starting points:

I am also happy to answer any questions you may have: claycloud.msg@gmail.com

Paul Graham: Distraction Kills Creativity

Recently, there was a Hacker News discussion entitled 'What habits of yours kill your creativity?'. A lot of people blamed 'distraction' for lost productivity.

In How to Lose Time and Money, Paul Graham writes:

The most dangerous way to lose time is not to spend it having fun, but to spend it doing fake work. When you spend time having fun, you know you're being self-indulgent.

Breaks from work are necessary, but try to avoid draining distractions. Instead, take a walk and disconnect.

 

Related Articles:

Paul Graham - Disconnecting Distraction

Chase Jarvis - Is Inspiration Killing Creativity?

Viget - Consumption: How Inspiration Killed, Then Ate, Creativity

Social Network Startup Idea: Better Filters

On the web, there exists a variety of 'filter failures'. Search engines such as Google are great for keyword search, but there are information needs for which keyword search is not ideal. When traditional search fails, leveraging a social network is often the solution to dealing with noise. For example, many people rely on Twitter for the latest news. Facebook helps people share photos and other content with their friends.

But even within social networks like Facebook and Twitter, there is noise: Not all content posted by the people you follow is interesting. It would be nice to have additional filters on top of social networks, and there is some work in this direction. Quora, for example, supports the ability to follow both people and topics. For Twitter, there is an academic project called Twitsper where you can send tweets to a restricted group. It will be interesting to see how social networks continue to evolve to address the noise problem.

Related:

Startup Experts: How to Achieve Eventual Success

Great, innovative business models typically aren't discovered on the first try. Flickr, Paypal, Twitter, Chegg, and Groupon are all examples of startups that successfully pivoted from initial ideas to their current business models.

How did they do it? Listed below are a few tips for improving the odds of eventual success:

1. Start Small

Starting small and releasing early gives quick feedback on what users really want. Andrew Mason, founder of Groupon, reflects on Groupon's predecessor, The Point:  

The biggest mistake we made with The Point was being completely encumbered by this vision of what I wanted it to be and taking 10 months to build the product, all the while making assumptions on what people want that we then spent the next 10 months backtracking on instead of focusing on the one piece of the product that people actually liked. You’re way too dumb to figure out if your idea is good. It’s up to the masses. So build that very small thing and get it out there and keep on trying different things and eventually you’ll get it right.

A smal feature set also makes it easier to do those features well. From Evan Williams' "Ten Rules for Web Startups":

#1: Be Narrow
Focus on the smallest possible problem you could solve that would potentially be useful. Focusing on a small niche has so many advantages: With much less work, you can be the best at what you do...This is all so logical and, yet, there's a resistance to focusing. I think it comes from a fear of being trivial. Just remember: If you get to be #1 in your category, but your category is too small, then you can broaden your scope—and you can do so with leverage.

Paul Buchheit has similar advice:

Pick three key attributes or features, get those things very, very right, and then forget about everything else.

2. Follow Your Passion

Energy is a huge competitive advantage. Marc Andreessen claims that the key to being lucky is simply having the energy to try many things: 

 In a highly uncertain world, a bias to action is key to catalyzing success, and luck, and is often to be preferred to thinking things through more throughly.

If you build a product that you care about, then you'll know whether it's good and how to improve it. Evan Williams states:

Great products almost always come from someone scratching their own itch. Create something you want to exist in the world.

It's hard to work on an idea if it's just about the money. From Michael Dell's advice to entrepreneurs:

You've got to be passionate about it. I think people that look for great ideas to make money aren't nearly as successful as those who say, "Okay, what do I really love to do? What am I excited about? What do I know something about? What's kind of interesting and compelling?"

The creation of great ideas can't be forced, and is often the result of a lot of hard thinking on related topics. Paul Graham states:

The way to get a big idea to appear in your head is not to hunt for big ideas, but to put in a lot of time on work that interests you, and in the process keep your mind open enough that a big idea can take roost. Einstein, Ford, and Beckenbauer all used this recipe. They all knew their work like a piano player knows the keys. So when something seemed amiss to them, they had the confidence to notice it.

3. Get Started

It is a mistake to wait too long for inspiration. Chris Dixon states:

I think a lot of people who are interested in starting companies think they shouldn’t do it until they have a Eureka moment.  I’d say that instead they should focus on finding an area that “feels interesting” and then get ready to bob and weave.

Finally, from Paul Graham's advice to high school students:

The important thing is to get out there and do stuff. Instead of waiting to be taught, go out and learn. Your life doesn’t have to be shaped by admissions officers. It could be shaped by your own curiosity.