In keeping with my nerdy addiction to homebrewed BI platforms, I made one using NVD3 and posted the code for you!

If you’re a fan of charts and visualizations, you’ve probably heard of D3. It’s great, but pretty powerful/complicated, and a downside is a lack of reuseable, simple api’s for common plots. NVD3 is Novus’ open source collection of reuseable d3 graphs, already designed and ready to go. Awesome.

Well, since I was in a dashboarding mood, I whipped together a prototype hack at a bootstrapped-out, responsive business dashboard application using nvd3 over a few hours scattered through my otherwise crazy last 2 weeks. Each chart’s business logic is fully encapsulated in a plot model which can be submitted to database manually now, but by webform in probably the next release. I’ve done a little bit of abstraction of the chart types, but so far only tested with line charts.

Check out the cool screenshot; you can find code in my github. https://github.com/rgehring/d3_fun

Want an entry level data science job? Chances are, you’re gonna need to be able to throw some pretty pictures onto a piece of paper; and these days, you might be expected to do it all on a whiteboard, in the interview! Lucky for you, RockStarStatistics is here to help. I’ve below outlined all the technical steps to rocking and rolling with some charts and dashboards!

1. Pick your metrics. I’ll post some more in-depth coverage of standard performance metrics as a follow up—be aware that your pointy-haired bosses favorite pastimes do include making up non-standard, poorly thought through performance metric ideas, so try to push back as much as possible during this stage and stick to what’s important. The very basics for online content are generally based around the impressions-clicks paradigm: you measure how many people saw something, and then how many of those people did something. You can get quite sophisticating about detecting patterns among the types of people who do the clicking.
2. Write a SQL append query to generate the results for the day. Typically you want to make a table like  ’agg_daily_analytics’ with the following schema:    ( day, demo_group, metric_a, metric_b, …  ). Your gonna have a bunch of metrics, write one subquery for each metric and work from the subqueries up. An example subquery could be:

SELECT COUNT(*) AS CLICKS FROM DIM_CLICKS WHERE CLICK_DATE BETWEEN CURDATE() - INTERVAL 1 DAY AND CURDATE()  

and another could be:

SELECT COUNT(*) AS IMPRESSIONS FROM DIM_IMPRESSIONS WHERE IMPRESSION_DATE BETWEEN CURDATE() - INTERVAL 1 DAY AND CURDATE()  

So, putting them together you’d have something like:

INSERT INTO AGG_DAILY_ANALYTICS (DAY, DEMO_GROUP, CLICKS, IMPRESSIONS) SELECT CURDATE()- INTERVAL 1 DAY, 'TOTAL', click_query.CLICKS, impression_query.IMPRESSIONS 

FROM (SELECT IFNULL(COUNT(*),0) AS CLICKS FROM DIM_CLICKS WHERE CLICK_DATE BETWEEN CURDATE() - INTERVAL 1 DAY AND CURDATE()) click_query 

CROSS JOIN (SELECT IFNULL(COUNT(*),0) AS IMPRESSIONS FROM DIM_IMPRESSIONS WHERE IMPRESSION_DATE BETWEEN CURDATE() - INTERVAL 1 DAY AND CURDATE() ) impression_query 

I didn’t actually syntax check the above but you get the idea. Make sure to handle the null case because it breaks cross join. Another option is to include the current date in each subquery and do a sequence of left joins on it — that way you can have null valued metrics.

3. Pick a web-based charting library. There are many, google charts is nice as is D3.

4. Copy the example code from your library of choice to display a graph of your data.

5. Put the SQL command to run your daily append scripts in the cron tab of your server so appends run nightly. Or use a fancy queue that restarts or blocks if jobs fail like celery.

Congrats, you’ve become a data scientist!

1. Get a consulting gig with a company you like.
2. Do what they hired you for.
3. Decide if you want to work there full time.

VS.

1. Survive an arbitrary interview process.
2. Blindly negotiate for a predefined role.
3. Accept or decline with poor information.

One of these is better than the others!

Now that I’m doing the ‘my-own startup’ thing, I figured I should start doing a bit more blogging. I’m not sure this qualifies as a ‘smart’ post or not, but I think design standards for websites have gone up lately… I’ve been wrestling with it and noticed a few patters, interspersed below with probably more information about my own problems than you’d care to know. (Enjoy the new site layout, by the way; although the decision to ditch my awesome burning guitar city header banner was an unrelated decision that’s been a long time coming! #professionalism). Anyway, on to the blog!

Just kidding—brief digression. So I’m cofounding a startup called GitSET. There ultimately will be some fancy hacker news blog post with some hopefully impressive stuff. For now suffice it to say that we’re turning Github into a sort of white pages for developers; we think we’ve solved people search for the open-source network. We finished the algorithms/back end/data science stuff necessary quite early in the startup process and have been focusing on monetizing and bd stuff in order to be able to finance our little adventure. However, I’ve wanted to build a web experience to showcase the awesome social potential of this data for a while. The only problem—lack of a designer/ front end guy on my team—leading me down to the DIY frontend design/css hell I’m sure so many of my readers call their day job. If you’ve spent a 6-hour stretch coloring buttons—I’m talkin to you!

At least to me, it seems like there are 3 points on a spectrum of pain that almost everybody finds themselves on.

• Poor design—typically results from having a data scientist or someone not artistically inclined be in charge of implementing the entire front-end experience!
• Overdesign—dressing up a questionable page concept in ridiculous over the top design hoping to win your eyeballs over with prettiness.
• Flat out #DesignDesperation — Ripping off other sites, or constantly monkeying with minor UX, as an escape from thinking about product at all!

I’m definitely dealing with the first one personally. Typically I learn best when my immediate circumstances shove my ignorance right in my face, forcing me to recognize that to accomplish what I need, I have to learn. I’ve always had that need for a little kick of fight or flight to really accomplish anything notable—and almost every challenge I’ve forced myself to look at this seriously has tended to work itself out in a hurry. Except for one—unfortunately, I’ve had a mental block around drawing and visual art since about the second grade when I’d eagerly trade writing the whole assignment for a 2-minute monochrome crayola illustration in any combo writing-drawing assignment. And maybe it’s because I’m bad at this, but wow, what a time suck illustrating pages is, even with bootstrap, assets, and a wireframe. I’ve never felt more deeply that I’m falling into the RUINATION of  ’I'll know it when I see it’ development than when designing visual elements solo. Seems like a truly orthogonal skillset and makes me appreciate all the great designers I’ve been privileged to work with who made this look easy at previous gigs!

For #2…. When I look around at the most fashionable startups, there’s this veneer of effortless elegance, like Steve Jobs and Michelangelo collaborated via time machine, caked onto even the shittiest, most pointless product concepts! And indeed, I feel that a certain level of design flash is now an expected part of the startup launch experience. Like Google’s patronizing , albeit vaguely educational #ParallaxPorn pop-out-storybook of an infographic explaining that search engines work by reading the internet, is emblematic of the typical publicity play exploitation of art. And it looks stunning.

Of course, a little excess can be appropriate from time to time, especially for companies not known for great design experience. But none of the above can compare to what I’m calling true #DesignDesperation. This is when your startup is failing and you can’t face it, so you waste your whole team’s energy away on silly design tweaks. Or you plagiarize other sites designs hoping it will somehow paper over the core deficiencies of your business model. Shoutout to Taskrabbit’s new Zaarly-esque look— and if you want more examples, try clicking through Crunchbase for an hour, because that stuff is pretty myriad. 

I guess I’ll cap this off  with the sort of half-assed conclusion that weak PM’s will continue to obsess over page design because it’s the only part of the experience they can viscerally understand and control, designers will continue to raise the bar across industry, and this will only cause increased heartache and business FAILURE  going forward for inexperienced managers. Recognize where you are on the spectrum, keep perspective on what’s important, take a shot, and ship.

As someone who’s been through RSI and pulled off a more or less full recovery, I get asked for advice on it a lot. Here’s what worked for me—your results may vary.

• Seek medical attention from a professional who specializes in hand injuries, as there are several conditions with fairly similar symptoms and different treatment strategies for each.
• Rest. Most forms of RSI have to do with inflammation. After intense overuse something gets injured, inflamed, and rubs up against the stuff around it. IE tendons in tendon sheaths. If you keep working while the tissue is still inflamed, it just gets more inflamed and more injured. Eventually, without correction you’ll get so injured work will stop one way or another. I wound up taking a month off typing completely.
• During rest, bracing. Keeping the wrist stable/UNLOADED gives time for inflammation to go down. Talk to a doctor / physical therapist about how long to rest in a brace vs. when to begin stretching, pt, etc. Also, don’t be one of those guys with a wrist brace who tries to overcompensate for it by being waaay too physically active, like throwing heavy books around and stuff. Yes, people do it, and yes, it’s dumb on many levels.
• Remember, it’s hardest at first and things will get better—I was at my psychological low when I felt I was ‘crippled for life’ and doctors hit me with bleak prognosis.
• Stretching / massage / general cardio health, to the extent you are able. Keeping bloodflow up promotes healing. You will be able to start PT during the rest phase.
• Eventually move to night bracing only, concurrent with gentle PT (stretches and eventually resistance) training to rehab your wrist, get muscles and blood back.
• When you return to typing work, do so slowly. Don’t work for a full hour. Some doctors may have you start with only 15 mins an hour typing allowed, although if you did a good enough job so far you may be already practically cured. But taking stretch breaks every hour is very healthy.
• Use an ergonomic keyboard to type (I like the kinesis advantage) and use dvorak if you can.
• Be aware of psychology — at the beginning of work resumption your mind may have learned an aversion to typing and you may feel pain that isn’t rooted in real physical distress. I know I did, big time! If this affects you, a useful mental technique is to think of all the reasons you may be stressed, and why your body may be reacting to this stress by giving you these pain signals. It’s very unlikely you would feel any pain after recovery like the above for a moderate workload unless your brain is tricking you!
• Return to weightlifting SLOOOWLY, but be aware you have a hand injury. For example, I can’t power lift or do pushups anymore, although I have aspirations to do pull ups again—but after months slowly building back up to it, I can now do equivalent lifts / loads on machines that keep my wrists in a stable, neutral position. So I do machine chest presses instead of pushups etc.
• Hand and extremity injuries are notoriously slow to heal. Don’t give up, if you take care of yourself things will look a lot better after a year.
• Some doctors will try and sell you on a magic pill to fix this — injecting you with anti inflammatories or giving you pills. These just treat the inflammation—really there’s no way around the fact that you have to give yourself time to heal and put in the work and effort to heal yourself.  Surgery, if appropriate, is also likely a ticket to at least a year of rehab—there aren’t really magic bullets.

Also, I say I’m cured, but I still do PT every day, take stretch breaks a few times every day, don’t play guitar or video games any more, and don’t try and do any handstands or similar wrist-intense stuff. In exchange for this good behavior I can type 8 hours a day without issue, pull the occasional 48-hour code frenzy, and live an otherwise normal life.

Okay, this may be a little ranty here, but my belief is this whole industry sector is like the amateur magic show some unfortunate family’s wayward uncle Gob puts on to ‘amaze’ the kids by pulling ribbons out of his puffy, overstuffed sleeves and flag waving to the final countdown. Or, for you techies, ASUS’s grand unveiling of the Padfone, a device offering all the processing power of a cell phone and the portability of a laptop.

I’m not sure who out there uses Asana, Basecamp, Glassboard, or similar project management software but this whole sector seems to have evolved to the level of sticky notes. I make a bullet point. Someone reads it. Maybe they put another one down or comment on it. Eventually it goes away. What the hell? This is 2012. Business collaboration should be reading my outlook, everyone elses outlook, doing analytics on that shit in the cloud and telling me who won the meeting last tuesday.

Maybe some of the pay solutions are better, but damn, the lack of quality free stuff totally blows my mind. Where is Github for business people?

Hey friends! You may have wondered why things got a little quiet for a few months there. Fear not! Things are awesome; I’m just crap at social media marketing. Here are some of the highlights from the past few months!

1) Zaarly 3.0′s to great fanfare and justifiably! Awesome product, the result of lots and lots of blood sweat and perseverance!

2) Ryan debuts on the nerdy tech scene by crashing multiple start up parties on boats, in historic buildings, and random bars that he was in several cases only accidentally invited to!

3) Solving interesting problems for work and pleasure, with secret nerdy side project science tba!

That about covers it for now… I’ll be sharing some new face-melting adventures and mind-bending paradigms sooner just as fast as you can whistle so stay tuned —same data time, same data channel!

Quick correction! From lookisg at the US national farm income data, the market rate for all of the US agricultural prodct is 262 BN expenses + 85 bn in profit. So dividing by 4 (we only need a quarter of the land) gives us ~90 bn; this is half a percent of us gdp and less than what we ALREADY spend on farm subsidies. Somebody elect me already!

Ok, if you were as scared shitless by THIS : http://www.youtube.com/watch?v=A7ktYbVwr90 as I was, you instantly went home and googled ways to fix global warming. Well, working tirelessly for 15 minutes with my trusty assistant google, I have already figured it out. And it will only cost us 10% of GDP!

Because I have absolutely no faith in the human ability to change habits that earn short term profits, I went looking for an engineered solution that trapped carbon. Preferably involving simple skills that people are good at already, like growing plants or lighting things on fire. Turns out that if you do those things, you can create charcoal, which is really great at trapping carbon.

http://www.popularmechanics.com/science/4297513 contains an expert estimate that if 2.5% of the world’s agricultural area were used to create charcoal, we could be down to pre-industrial levels of CO2 in 40 years.

So how much would it cost? I googled the price of agricultural land in the US. We’d need about 1 MM square km to hit the magic 2.5% number; that’s about a quarter of the farmable land in the US. At $3k per acre, thats about $750 bn — 5% of US gdp — to buy the land with a nice premium over market price. The price of farming the land will run about $150 bn every year thereafter according to current government measures.

So then all we need to do is char up the harvest, maybe wrap it up in plastic wrap if we want to be extra safe, and bury the resulting charcoal deep in the earth. Done and done.

It might even be possible to bring the price down if we harvested the needed biomass from wild or fallow land, but I couldn’t be assed to look up the numbers on that one.

So people, fire your congress and put me in charge of the economy! You’ll get enough new, green jobs to put america back to work solving the greatest extinction threat humanity has ever faced, except perhaps nuclear holocaust, which by the way I am also opposed too.

Just for perspective: how much money is $750 bn, REALLY? About one quarter of the price of war in Iraq, Afghanistan, and Pakistan. So if we could just invade one less country over the next ten years, we’d already come out ahead.