In San Francisco, Startup == Pub

Any event where you wake up the next morning with two free shirts, a deck of cards, a jar of bubble soap, a stack of business cards, and a video of yourself ziplining across an office hallway is a resounding success. Enter Startup Crawl, an open house on some of San Francisco's hottest startups. It's part pub crawl, part power-speed-networking, part tech-speak geekout session, and 100% SF-style general hilarity. Where else can some lowly Canadian undergrads walk in off the street - in jeans and T-shirts, no less - and end up throwing back Don Julio with the CEO over talk about scalability and the startup life within the half hour? Where else can you climb out a window to have rooftop beers with high-wired entrepreneurs preparing for next morning's Y Combinator pitch? The list of such meritocratic hedonist enclaves is short indeed.

This event deserves to be ported elsewhere - where's the beef, Waterloo?

Mean Streets of Silicon Valley

Mean/variance calculation is ridiculously commonplace in data analysis, yet most programmers have never seen this gem from TAoCP:

def online_mean_and_variance(data):
    n, mu, s2 = 0, 0, 0
    for x in data:
        n += 1
        delta = x - mu
        mu += delta/n
        s2 += delta*(x - mu)
    if n > 1:
        yield (mu, s2/(n-1))

Unlike the standard two-pass algorithm, this one is online; it also happens to be more stable.
If that's not enough, I've given it to you here as a Python generator. Enjoy!

iPwned

It was only a matter of time before Google did this - let the battle begin.

802.11x In The City

Another morning, another wireless hotspot courtesy of the thriving West Coast coffee-shop culture. This one is located within the confines of Cup Of Java, a fairly standard joint that just so happens to have a Casio CTK-471 sitting around - any instrument on the keyboard-piano spectrum, even a cheap MIDI-spewing plastic-encased Casio, earns bonus points. I order the Greek Omelet, which comes nestled between a bed of -

Actually, never mind that. I'm no food critic; that's what we have Yelp for. (Hooray for externalized memory!)

In San Francisco, every coffee shop comes with its very own resident Bluetooth headset-wearing intravenous-caffeine-drip work-from-home maven; the more popular ones boast entire armies of these laptop-toting soldiers, each one vying for a slice of the ether. (I'm taking more than my fair share with Transmission (more proof that the highly publicized takedown and subsequent sale of The Pirate Bay does jack-all to staunch the, er, torrential (yeah, I winced too, but it seemed like most appropriate adjective) flow of information), and am watching nervously over my shoulder for the legions of RIAA men-in-black that are no doubt waiting to haul my recidivist pirate ass into court.) I'm half tempted to whip out Wireshark to see what exactly these people are doing, but that would most likely cross the line between harmless curiosity and feline genocide.

...and I'm off to scour a couple of thrift stores before grabbing the bus into work.

The Gubernator Strikes Back

Need one more reason to be proud of your elected officials? Here you go. While we're at it, here's another epic fail courtesy of our beloved civil engineers.

What else? I went ATVing at the Pismo Dunes State Vehicular Recreation Area, an unnecessarily obtuse moniker for what amounts to a giant outdoor sandbox filled with off-road vehicles of every description. As a testament to its rugged nature, the area is rife with Second-Amendment freaks on oversized rollcages-with-wheels proudly festooned with Southern Confederate flags. If you've ever wanted to live out those Mad Max post-apocalyptic fantasies of petrol-fueled villainy, this is most definitely the place.

Found a super-chill haven of hippie-dom sandwiched between Mission and Valencia. The Oz Hookah Lounge delivers drunk revelry, multiflavoured smoke haze, cushions, and a panoply of psychedelic patterns at nearly every hour of the night. This particular visit was punctuated by a host of inebriated Iranian expats carrying beer-laden ice buckets and gin-and-tonics from a nearby bar, which they promptly distributed amongst the crowd.

Halloween's coming - get your costumes together! Photographic evidence is requested, lest the veracity of your respective accounts be called into question.

Eye of the Sloth

It's training time - my general fitness took a proverbial beating from the demons of stress, work, and take-out cuisine last term, and I'm just now starting to rectify that. (Of course, there's no snow here, so I'll have to settle for Cali sun and SF fog. Woe is me.) To that end, I'm making a commitment to hit the gym at least tri-weekly; encouragement (and light berating, if that's more your style) is, er, encouraged. I'm hoping to make it out to some of the SFPK meets as well. Naturally, I'll be concocting my own personal brand of inspiration - if this cryptic remark has you puzzled (as well it should) you'll just have to keep posted for more details!

Putting the Ignoble in Vignoble

Had the chance to wine and dine in the famed wine-producing region of Napa Valley this last Saturday. I decided to don my only suit jacket (and no socks - hey, it's laundry day) for the occasion so as to meet the requisite minimum standard of pomposity. (I should really source a monocle, kerchief, pocket watch, and top hat. As a general rule, you don't argue with anyone who is actively employing all these objects in the service of highbrowdom.)

The Stanford Canadian Club Thanksgiving dinner was a resounding success as measured by the excess of available foodstuffs; there was turkey, stuffing, mashed potatoes, and pumpkin pie in abundance. Yum. Maybe this whole gastronomic shindig should be a weekly affair (or fortnightly, at the very least.) In true seasonal fashion, the leaves are Monty-Python-style suicide-diving off the trees in droves; the air is crisp with that familiar decay-scent, something I had not expected to find this far south.

Important news flash: I just saw a rainbow out the shuttle window. If I were inclined to believe in such nonsense, I'd say that was a good omen for the day.

My One Lunar Cycle No-Post-iversary

Yeah, it's been that long. 28 Days Later, stuffs and things at varying orbitals of excitement have been happening with acceptable regularity:

• Not only am I the proud owner of a longboard, but this particular plank of my comprehensive transportation platform now bears the inimitable insignia of one Randall Munroe. (I can only wonder what joyous blasphemies #666 will bring.)
  
• Finally made it out to the infamous DNA Lounge; this SF institution has the peculiar distinction of being owned by ex-Netscape programmer Jamie Zawinski.
• Canadian Thanksgiving shindig, complete with full turkey and pumpkin pie. Need I say more? (No, but I should probably add this: there's another Canadian Thanksgiving dinner this Sunday hosted by the Stanford Canadian Club; despite what their (apparently infrequently updated) site says, it really is this weekend.)
  
• Pretty Lights at The Independent last night. If you haven't seen them, you really should.

How's Facebook? (Come on - if you care enough about me to read my pithy musings, you're probably expecting an answer to this question.) Intense. It pains me to say it, but Facebook has so far provided much more in the way of personal development and work-related awesomeness than Google ever did. How can that be? It comes down to recruiting strategy: Google casts as wide a net as possible, hoping to grab what it considers to be the best of the best before someone else does. In my opinion, this strategy is bound to backfire. You end up with a zillion interns and, well, less than a zillion interesting and/or useful projects; it doesn't take a math major to see that you can't pair each intern up with something worthwhile to work on. (It does take a math major, however, to look at the whole situation and start rambling about bijections.) Here's the point: half of the interns coming out of Google are extremely wary of returning, and that can't be good. ("Half" here is an extremely unscientific guesstimate, but several co-interns (who will obviously remain nameless in such a public forum as The Internet) have expressed similar sentiments.)

The Pirates United Will Never Be Defeated

"We have, ourselves, full confidence that if all do their duty, if nothing is neglected, and if the best arrangements are made, as they are being made, we shall prove ourselves once more able to defend our Internets, to ride out the storm of war, and to outlive the menace of tyranny, if necessary for years, if necessary alone.
Even though large parts of Internets and many old and famous trackers have fallen or may fall into the grip of the Ifpi and all the odious apparatus of MPAA rule, we shall not flag or fail. We shall go on to the end, we shall fight in France, we shall fight on the ef-nets and darknets, we shall fight with growing confidence and growing strength in the air, we shall defend our Internets, whatever the cost may be, we shall fight on the beaches, we shall fight on the baywords.org, we shall fight on the /. and on the digg, we shall fight in the courts; we shall never surrender, and if, which I do not for a moment believe, the Internets or a large part of it were subjugated and starving, then our Empire beyond the seas, armed and guarded by the Anon Fleet, would carry on the struggle, until, in Cerf's good time, the New World, with all its power and might, steps forth to the rescue and the liberation of the old."

Onwards and Upwards

Congratulations, Facebook! Now for the other 6.4 billion.

(Side note: looks like Google's playing Wolfram Alpha's game. Round Two: fight!)

The Ongoing Saga of Evan's Search For Housing in San Francisco

I've finally uploaded a couple of albums, as promised - one for our intern-squad trip to LA, one for random.choice() style samplings of SF. The latter album will hopefully expand as the term progresses!

The small handful of people that actually read this thing might have noticed a considerable lull in post volume - and not without good reason. For illustration, here's a timeline of my weekend:

• Friday, 9 pm: pass out from exhaustion. Turns out I still haven't caught up on sleep from the month of death, a fact which late-night mid-week meanderings in Mission can only exacerbate.
• Saturday, 9 am: wake up, foot it to Lower Haight for breakfast.
• Saturday, 11 am: still at breakfast place, firing salvos of Craigslist replies. Aiming for the clusterbomb strategy; the housing search in San Francisco is not exactly easy, especially when you're a male international student looking for something short-term.
  
• Saturday, 12 pm: moved on to Upper Haight, where I score myself a longboard.
• Saturday, 1 pm: running laps around here to the dismay of tourists and local art cognoscenti alike. Still need practice before I can take this thing to the streets.
• Saturday, 2 pm: finally work up enough confidence to book it down to Stockton and Market, where I've got an overpriced screen-repair appointment at the not-so-aptly-named Genius Bar.
• Saturday, 4 pm: with everything in working order, I work my way back to the hostel.
• Saturday, 5 pm: keycard doesn't work. Uh oh.
• Saturday, 5:30 pm: I learn that, contrary to my understanding, the hostel has fubar'd my reservation; I was supposed to check out this morning. At least they're nice enough not to charge me for the privilege.
• Saturday, 6:50 pm: after a hasty sack-packing and another ear-grinding leg on the BART, I'm standing on the San Jose-bound platform at Millbrae Caltrain. The plan? Lug myself to the office, drop my stuff off, and find a nice comfortable couch to crash on. Oh, and I just missed the train.
• Saturday, 8:30 pm: I finally get to the office.
• Saturday, 9 pm: I check my email, where I find a welcome bit of good news - one of the Craigslist postings actually responded.
  
• Saturday, 9:30 pm: One of my coworkers happens to be working late on a rush job; he kindly offers me a place to crash for the night.
• Saturday, 11 pm: I'm watching Clash of the Titans. Hey, this nomad business isn't half bad.
• Sunday, 11 am: me, my longboard, and I make it out to the Caltrain for a trip back into SF. Not having a place to stay, I ping jverkoeyen; he volunteers the services of his fine floor for the cause. Random discovery - turns out there's a farmers' market right outside California Ave. Caltrain every Sunday.
  
• Sunday, 2 pm: standing at Embarcadero and Market. The Craigslist poster rings me; I've arranged a viewing at 4 pm. Between then and now, I've got to find Jeff and get my ass across most of downtown SF to the far side of Mission. Whatever will I do?
• Sunday, 3 pm: turns out Jeff and his girlfriend Emily shelled out for a mattress and bedframe off Craigslist and need help moving it. In return, they'll ship me across town in a Zipcar pickup truck and pass off a set of keys so I can get into their place.
• Sunday, 3:40 pm: I reach the room I'm viewing with time to spare. The tenants show me around. Good vibes all around; these people are exactly the kind of laid-back I need right now.
• Sunday, 5 pm: eating in some supposedly Cambodian restaurant near Mission.
  
• Sunday, 8 pm: a meal, a spot of rainy longboarding, a none-too-quick bus ride, and a short hike later, I reach Jeff's place. It's exactly as sparsely furnished as I had imagined.
• Sunday, 9 pm: I find a corner supermarket with the help of a map Jeff left on the counter.
• Sunday, 9:30 pm: I'm eating a hasty meal of baguette, cheese, turkey, pasta salad, nectarine, and orange juice under Bay Bridge.
• Sunday, 11:30 pm: restless, I take a brief promenade about the apartment complex grounds. Nice pool; there's something pleasantly surreal about living in the shadow of an enormous bridge.
• Sunday, 12 am: lights out. Too much busy-work roaming for one weekend.

And that was it. Gasp. I must have covered the whole city a zillion times.

On the upside, I've landed a place to live for the rest of the term; I'm bunking up in my favourite hostel in SF until move-in. That, and I have yet to injure myself longboarding. (Depending on who you ask, that's either a good sign or an indication that I'm not doing it right. We'll see - I've got plenty of time and hills to reverse this trend with.)

Board in SF

Shelled out a wad of cash today for a longboard, which I've been (painlessly (so far, at least)) riding around SF on. My goal is to cut door-to-door times in the city core by 60-75% compared to the estimated walking time on Google Maps. (I'd estimate 80-90% for cycling, but this is immeasurably cooler - not to mention far more prone to catastrophic failure.)

And, yeah, I promised pics a few days back. I have yet to deliver on that promise, but I'll definitely post here when I do!

You've Been Facebook'd

Where "you" is TechCrunch. Long live engineer humour!

Where Two Files Meet

(Note: I promise that I'll post something more generally interesting before this day draws to a close, for non-technical values of generally interesting.)

Let's say that you have two large unsorted files, each of which is essentially a long list of strings, and you want to find their intersection. There's the simple brute-force way:

while read line; do grep "$line" file2; done < file1

There's a problem - this is quadratic! Fortunately, we can cut this down pretty easily with the old time-memory tradeoff:

sort file1 > file1.sorted

sort file2 > file2.sorted

sort -m file1.sorted file2.sorted > combined

diff combined <(uniq combined)

Denouement

"It is criminal to steal a purse, daring to steal a fortune, a mark of greatness to steal a crown. The blame diminishes as the guilt increases." -- Schiller

I was sitting in the City Cafe (accent intended) Bakery (held in slightly-above-average esteem by the professor who brought Waterloo's CS frosh such delights as Scheme), taking a quick breakfast with my brother; this quote was written on their blackboard. City Cafe is an economic enigma. They have no cashiers, opting instead for a self-serve honour-system model; you deposit your payment into a bus fare box (Cleveland Transit, I'm told, although I've never verified this) before leaving.

Question of the day: why does this work? By traditional economic theories, the dominant strategy is to eat and run. One theory holds that this is a form of the tit-for-tat strategy: the City Cafe Bakery places trust in its customers, who then feel compelled (perhaps because reciprocity helps maintain social norms) to pay. Another possibility is that the players are maximizing self-interest according to a different payoff function. There might be quantifiable reward in keeping the bakery open or in appearing honest to one's friends. (It is notable that customers often leave tips - maybe it is psychologically easier to tip when the register and tip jar are combined into one receptacle!) There is also the novelty factor; by following an unorthodox business model, they generate a certain amount of word-of-mouth buzz.

(According to Schiller, your best strategy is to grab everything in the bakery and run - that way, you minimize blame!)

Denouement (again, accent intended) comes from the French noeud for knot; the literal translation is "unknotting". I'll leave that open to interpretation.

Context-Free Blogging

My hobby: thinking up weird pseudo-science-sounding phrases and investigating the resulting Google harvest. This time around, a search for context-free schooling (thought process: take CS 360, add various ruminations on the state of modern education, bring to simmer) returned a list of links to this paper on the impact of education on contextual information processing. (The original absolute/relative tasks are described in greater detail here.) The question of how much you can really extrapolate from drawing lines in boxes is left open; in any case, we may be able to include big-picture connection-building among the victims of traditional schooling.

Oh, and the Internet has finally invented meta-time-wasting. Enjoy!

The Eleventh Hour

Aside from being the title of an awesome children's book (which may or may not still reside on my shelves), this phrase adequately sums up my situation here. I've got slightly under 72 hours to polish and document my Graphics project, touch up an essay on Church's solution to the Entscheidungsproblem of Hilbert's Program, and - of course! - push the trains struggle as far as possible.

(Yes, I know you can see the edges of the skybox in the above image. I probably won't have time to change that before the Graphics demo.)

On the real-time front, we've had a very fortunate breakthrough; the sensor polling bug fell before a quick debugging sprint yesterday morning. We're driving hard to get route-finding working tonight, as this will leave us all of Tuesday (as mentioned, I have other things to worry about tomorrow!) to put together our final demo. Given the time constraints, we'll likely have to go with something almost braindead-simple; the plan currently being floated around is to hack together a cat-and-mouse game using the trains.

But enough about schoolwork! I'm off to Germany on the 30th. Let's hope I manage to drag my bedraggled body to Pearson in time!

Final note: this blog has been very one-dimensional over the last month. Given my situation, this is perhaps understandable; nevertheless, I shiver with antici-

Delay(60);    // pause for 3s (one tick == 50 ms)

-pation at the prospect of having something other than Graphics or Real-Time to talk about.

Ore for Wood

5 days to go - I've missed a few here. Graphics is going reasonably well; the past day or so has seen the addition of skyboxes, proper terrain clamping, texture blending on the terrain (albeit a hacked-together software version), smooth normals, water reflection, and a few test scenes to my project. Above: a test scene that shows the use of the skybox (rendered in garish colours for maximum obviousness!), one tree (with eight levels of branching), and a few randomly positioned rocks (two levels of subdivision each).

Real-Time, on the other hand, is coming to an inauspicious close. Our program suffers from a showstopper bug: sensor polling freezes. We managed to hack around this to complete most of the first train control objective, but the second control objective will remain elusive until this is resolved. The PsychOS team is running out of time rapidly, but we still have a few more tricks up our sleeves:

• We can redesign sensor polling. The current plan is to extend the kernel's event-handling structures to support software events as well; this way, the SensorManager() can trigger EVENT_NEW_SENSOR (or something like that) whenever a new sensor comes along.
• We can attempt adding a chain of couriers (or similar data-passing tasks) to the train input server. If the problem lies in dropped bytes, this might resolve the issue.
• Failing all else, we can add some NOPs (or the C equivalent: a busy-wait for loop - our TA actually did this when he took the course.) The mere thought of adding such a blatant hack induces violent fits of cringing, but we'll do it if we need to.

If we can get past this, we already have systems in place to find routes and reserve track sections - they just need to be tested and used.

Personal notes: it is now quite obvious that the month is drawing to a close. This has been a bizarre ride of ups and downs, of hope and despair, of sleeplessness and take-out meals and 16-hour debugging sprints. I'm feeling strangely calm as I enter the final stretch, in stark contrast to the near-complete-breakdowns that some of those Real-Time sprints drove me to. Not even the possibility of missing the final Real-Time demo fazes me; regardless of what happens with the project, I've learned an incredible amount about hardware, low-level programming, and operating system design - and it's all practical knowledge, especially given that we've been working with the ARM architecture.

If I had to pinpoint the source of our problems, I'd put it down to hubris. The early assignments were relatively easy, the test programs rather trivial; we simply wrote the required code and then went off to take care of our other assignments, working on the assumption that everything would continue to go well. This is never good practice, not even in the classroom. We implemented proper error-checking about three weeks ago, something we should have done three months ago. We should have rigorously tested our kernel, bombarding it with interrupts and I/O and requests and everything else that could possibly bring it to its knees; we did some of that, but it was too little and far too late.

Despite the above tirade - which looks suspiciously like a pre-emptive post-mortem - I should add that I refuse to give up. Once I get Graphics out of the way, we'll dig our heels in this weekend to try anything and everything that might work. Keep posted!

Trees!

10 days to go. After plowing through the first few sections of Prusinkiewicz and Lindenmayer's The Algorithmic Beauty of Plants, I hacked together a quick tree renderer. The above image was generated by randomizing the parameters in Figure 2.8, with slightly higher branching angles to fake downward tropism. (Yeah, I know that's not how it's really done - but it's dead simple.)

At this point, I need to start trying to mash things together. I could spend several days tweaking my texture, terrain, rock, and plant generators to near-perfection, but I just don't have the time - especially when I've also got to slap in texture mapping, a skybox background, a simple water plane with stencil buffer reflection, interaction (I've got the keyboard working, but I haven't implemented mouselook yet), terrain clamping, and collision detection.

Phoenix Down

11 days to go. No picture this time, but I don't need one: we fixed the kernel, thus reviving ourselves just in time for the final sprint. Depending on how the next couple of days go, we may even be able to make the Train Control 2 deadline on Monday. This milestone - the last before the final project - requires groups to track the motion of two trains while preventing collisions.

Graphics front? Nothing for today. I'll be putting in a sprint tomorrow, which will hopefully furnish me with more flashy images to post here on qx5. I'm treating myself to a break this evening; given that I've been putting in 10-14 hour workdays every day including weekends this fortnight prior, such a well-needed pause could prove instrumental in averting catastrophic burnout.

(For the more observant reader: yes, the days-to-go counts over the last three posts have been inconsistent. This one is correct; I'm counting down to the Graphics project deadline. I'd wish myself good luck, but dtam says luck is for chumps...so I'll wish myself good skill instead.)

Well, I'm off to go eat pizza and watch Bill Nye with our friendly neighbourhood CSC. Keep posted!