"pv – Pipe Viewer – is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion." Looks very handy.
"xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files. Commands should be simple, fast and composable." iiinteresting.
OK, this is great: Bret Victor's library for exploring interactive documents. Tidy – thanks to its use of data-attributes – but super-clear. Really nice to have a web-based library, too, and one focused on text. Now thinking about this conceit again.
"There was also one late night when a stranger opened the door and walked into the house when August should have auto-locked the door. (The stranger was trying to enter our next-door neighbor’s house and didn’t realize he was at the wrong door.)" YOU HAD ONE JOB etc.
Rather looking forward to seeing this play out: thirty days of processing and spelunking CSV, from Paul Downey. Lots of new tools and tricks emerging already.
Really nice exploration of a small stack for poking data at the commandline. I'm a fan of jq and its ilk already, so this extends some of those techniques.
"All it takes to get a website going for a repository on GitHub is a branch named gh-pages containing web files. You also don’t need a master branch, you can have a repo with just one branch named gh-pages. Here is what I think is really cool, if you fork a project with just a gh-pages branch, you’re only a commit away from having a live version yourself. If this repo being forked is using sheetsee.js then everyone is a fork, commit and spreadsheet away from having a live website connected to an easy (a familiar spreadsheet UI and no ‘publish’ flow because Google autosaves) to use database that they manage (control permissions, review revision history)." Very smart.
Hosted statistics tool with attractive interface and smart API. Not cheap for its single-tier plan ($99/mo), but looks like it might be worth a poke.
"Here we get a glimpse of an alternative figuration of data itself. Rather than some kind of precious (but immaterial) stuff, or fuel for market speculation, data here is a relationship, a link between one part of the world with another, and a trace that can be endlessly reshaped."
"DataFart lets you easily graph data from the command line." So it does.
22 November 2012
The work is in parts a technology prototype, data visualisation, and artwork. Custom-built, open-source software is used to analyse performance video and generate plots of actors’ positions on stage from a perspective viewpoint. These plots are then used to generate new, secondary artworks: posters, and laser-cut wooden shapes.
The project emerged from an initial workshop and commission by Caper, where we explored various potential ways for technologists to collaborate with the RSC on short projects. From there, I dealt with the RSC direct, meeting key members of their team and understanding a bit more about the various factors influencing performances and productions there.
It was great to be able to take such a fluid, interpretative approach to the work. With hindsight, this was unsurprising: the RSC’s business is interpretation – taking Shakespeare and producing entirely new productions each year, of plays they have often performed countless times. My work was similarly interpretative: initially, building software to explore the data, and then exploring that data as a material – before moving onto the further material exploration of output formats. It’s the sort of structure to work that I’m fond of.
It was also great to have a brief to shape, and ultimately push myself: not just exploring a single technical idea, but seeing it through, end-to-end, to output and display. It was important to me that whatever came out of it – however prototype-y – was both beautiful and accessible. I think the output – especially the lasercuts – has stood up to that internal demand.
Thanks to Rachel and Kat at Caper for setting up the initial commission and the workshops; to Sarah and Ida, for producing the work from the RSC so superbly; and to everyone I met at the RSC who offered insight, ideas, and knowledge.
And, if you’d like to know more about it, or indeed, to work with me on similar work – be it investigative, creative, or artistic – do get in touch.
"The moment that stopped me in my tracks was when I checked to see if there was anything in the external disk drive." I really want to find out what's on it. Lovely, simple storytelling from Aanand.
29 June 2012
Max has finally written the brilliant article about real-time data visualisation, and especially football, that has always felt like it’s been in him. It’s in Domus, and it’s really, really good.
This leapt out at me, with a quick thought about nowness:
Playing with a totaalvoetbal sense of selforganisation and improvisation, the team’s so-called constant, rapid interchanges — with their midfielders often playing twice as many passes as the opposition’s over 90 minutes — has developed into a genre of its own: “tiki-taka”. Barça have proved notoriously difficult to beat, and analyse. Tactical intentions are disguised within the whirling patterns of play.
It serves as a reminder of the special power to be gained from resisting analysis, of being unreadable. Resisting being quantified makes you unpredictable.
Or, rather, resisting being quantified makes you unpredictable to systems that make predictions based on facts. Not to a canny manager with as much a nose for talent as for a spreadsheet, maybe, but to a machine or prediction algorithm.
This is the camouflage of the 21st century: making ourselves invisible to computer senses.
I don’t say “making ourselves invisible to the machines”, because poetic as it is, I want to be very specific that this is about hiding from the senses machines use. And not to “robot eyes”, either, because the senses machines have aren’t necessarily sight or hearing. Indeed, computer vision is partly a function of optics, but it’s also a function of mathematics, of all manner of prediction, often of things like neural networks that are working out where things might be in a sequence of images. Most data-analysis and prediction doesn’t even rely on a thing we’d recognise as a “sense”, and yet it’s how your activities are identified in your bank account compared to those of a stranger who’s stolen your debit card. Isn’t that a sense?
The camouflage of the 21st century is to resist interpretation, to fail to make mechanical sense: through strange and complex plays and tactics, or clothes and makeup, or a particularly ugly t-shirt. And, as new forms of prediction – human, digital, and (most-likely) human-digital hybrids – emerge, we’ll no doubt continue to invent new forms of disguise.