dialectical futurism

Covered In LaTeX

2010-08-31T00:00:00-04:00

Although I haven’t used LaTeX much in the past few years, it was one of the primary tools that hastened my shift to using GNU/Linux full time. Why? I’d grown sick of fighting with document preparation and publishing systems (e.g. Microsoft Word/Open Office), and had started using LaTeX on my Mac to produce all of my papers and documents that needed to be output to paper-formats. Why switch? Because after a certain point of having every tool you use be Free software (because it’s better!), it becomes easier and more cost effective to just jump the gun and by commodity hardware and use a system that’s designed to support this kind of software (managing a large selection lots of free software packages on OS X can become cumbersome).

So why LaTeX? What’s the big deal? Why do I care now? Well…

LaTeX is a very usable front-end/set of macros for the TeX typesetting engine. Basically, you write text files in a particular way, and then run LaTeX (or pdflatex) and it generates the best looking PDF in the world of your document. You get full control over things that matter (layout, look and feel) and you don’t have to worry about things that ought to be standard (titles, headlines, citations with BibTeX, page numbering, hyphenation). The best part, however, is that once you figure out how to generate a document correctly once, you never have to figure it out again. Once you realize that most of the things you need to output to paper are in the same format, you can use the same template and be able to generate consistently formated documents automatically. There’s a “compile” step in the document production process, which means changes aren’t often immediately recognizable, but I don’t think this is a major obstacle.

Word processing and document preparation is a critical component of most common computer users. At least, I’d assume so, though I don’t have good numbers on the subject. In any case, I think it might be an interesting project to see how teaching people how to use LaTeX might both improve the quality of their work, and also the way that they’re able to work. It’s advanced, and a bit confusing at first, but I’d suspect that once you got over the initial hump LaTeX presents a more simple and consistent interface: you only get what you tell it to give you and you only see the functionality that you know about. This might make the discovery of new features more difficult, but it doesn’t limit functionality.

I’m not sure that this post is the right space to begin a lesson or series on getting started with LaTeX, but I think as a possible teaser (if there’s interest) that the proper stack for getting started with LaTeX would consist of:

A TeXlive distribution. You need the basic tool kit including pdflatex, TeX, Metafont, LaTeX, and BibTeX.
A Text Editor with LaTeX support: emacs, TextMate, etc. Plain text can be difficult and cumbersome to edit unless you have the right tools for the job, which include a real text editor.
Some sort of macro or snippet expansion system. TeX is great. But it’s also somewhat verbose, and having an easy way to insert text into your editing environment, both for templates but also for general operations (emphasis, notes, etc.) is incredibly useful, and reduces pain greatly.
A template management system. This probably needn’t be a formal software system, but just something to organize and store the basic templates that you will use.

And the rest is just learning curve and practice. Onward and Upward!

Console Use for the Uninitiated

2010-08-13T00:00:00-04:00

I have a computer, an old laptop, that I mostly use as the foundation of my stereo system. It’s just a basic system with a few servers (a web server and the music player daemon), and it doesn’t have a running window manager. This configuration usually doesn’t bug me: I connect remotely and the computer sits under the couch, but since my recent move I’ve not had a network connection at home and I’ve defaulted to playing music and managing the system from the console.

This works just fine for me. The virtual terminals aren’t noticeably different from the terminal I get over ssh (as you would expect/hope), except now I have to walk across the room. The people who listen to music with me haven’t yet been other terminal geeks, and so I’ve taken on the role of stereo whisperer. Until a friend looked over my shoulder and wanted to change the track. Using the console is sometimes (often) a slippery slope.

I realized immediately that this situation was much more conducive to learning to use the console than the kinds of introductions to using the console that I’ve typically written. The commands we used were very limited: the mpc program that acts as a simple command-line client to the music player daemon (mpd) and grep. We also used the pipe operator.

There are thousands of commands on most Linux/UNIX systems and remembering all of them can be a bit challenging. The console is a limiting environment basically you can do one thing at a time with it, and you don’t have a lot of leeway with common errors. At the same time, there are a great number of programs and commands that a beginner has no way of knowing about or knowing when to use. Legitimately, the console is both too limiting and expansive to be quickly accessible to the uninitiated. Starting with a very limited selection of commands is way to break through this barrier.

The terminal environment is also very “goal-oriented.” Enter a command to generate some sort of output or result and then repeat. At the end your system will have done something that you needed it to, and/or you’ll learn something that you didn’t already know. When you’re just trying to learn, all of the examples seem fake, contrived, and bothersome because many people already have an easy way of accomplishing that task using GUI tools. Learning how the terminal works, thus, needs a real example, not just a potentially realistic example.

The great thing, I think, is that once you have a need to learn command line interaction, it makes a lot of sense even to people who aren’t die-hard geeks: Commands all have a shared structure that is fairly predictable and inconsistencies are apparent. Perhaps most importantly the command line’s interaction model is simple: input a command and get a response. Advanced users may be able to bend the interaction model a bit, but it is undeniably parsimonious.

It seems, in conclusion, that the command-line is easy to learn for the new user for the same reason it is beloved by the advanced. Ongoing questions, include:

If this kind of realization were to catch on, how might it affect interaction design in the long run? Might “simple to design” and “easy to use” move closer together?

Is there a way to build training and documentation to support users who are new to this kind of interaction style?

Collaborative Technology

2010-08-05T00:00:00-04:00

I agreed to work on an article for a friend about the collaborative technology “stuff” that I’ve been thinking about for a long time. I don’t have an archive that covers this subject, but perhaps I should, because I think I’ve written about the technology that allows people to make things with other people a number of times, though I have yet to pull together these ideas into some sort of coherent post or essay.

This has been my major post-graduation intellectual challenge. I have interests, even some collected research, and no real way to turn these half conceptualized projects into a “real paper.” So I’ve proposed working with a friend to collect and develop a report that’s more concrete and more comprehensive than the kind of work that I’ve been attempting to accomplish on the blog. Blogging is great, don’t get me wrong, but I think it leads to certain kinds of thinking and writing (at least as I do it,) and sometimes other kinds of writing and thinking are required.

Regarding this project, I want to think about how technology like “git” (a distributed version control system) and even tools like wiki’s shape the way that groups of people can collaborate with each other. I think there’s an impulse in saying “look at the possibilities that these tools create! This brave new world is entirely novel, and not only changes the way I am able to complete my work, but how I look at problems, and make it so much easier for me to get things done..” At the same time, the technology can only promote a way of working it doesn’t necessarily enforce a way of working, nor does any particular kind of technology really remove the burdens and challenges of “getting things done.” More often perhaps new kinds of technology, like distributed version control, is responsible for increasing the level of abstracting and allowing us (humans) to attend to higher order concerns.

Then, moving up from the technology, I think looking at how people use technology in this class allows us to learn a great deal about how work is accomplished. We can get an idea of when work is being done, an idea of how quality control efforts are implemented. Not only does this allow us to demystify the process of creation, but having a more clear idea of how things are made could allow us to become better makers.

The todo list, then, is something like:

Condense the above into something that resembles a thesis/argument.
Become a little more familiar with the git-dm (“data mining”) tool that the Linux Foundation put together for their “state of Kernel development.”
Develop some specific questions to address. I think part of my problem above and heretofore has been that I’m saying “there’s something interesting here, if we looked,” rather than. “I think w kind of projects operate in x ways, where y projects will operate in z ways.”
Literature review. I’ve done some of this, but I’ve felt like I need to do even more basic methodological and basic theory reading. And even though an unread Patterns of Culture is on my bookshelf, I don’t need to read that to begin reading articles.

That’s a start. Feedback is always useful. I’ll keep you posted as I progress.

Saved Searches and Notmuch Organization

2010-07-30T00:00:00-04:00

I’ve been toying around with the Notmuch Email Client which is a nifty piece of software that provides a very minimalist and powerful email system that’s inspired by the organizational model of Gmail.

Mind you, I don’t think I’ve quite gotten it.

Notmuch says, basically, build searches (e.g. “views”) to filter your email so you can process your email in the manner that makes the most sense to you, without needing to worry about organizing and sorting email. It has the structure for “tagging,” which makes it easy to mark status for managing your process (e.g. read/unread, reply-needed), and the ability to save searches. And that’s about it.

Functionally tags and saved searches work the way that mail boxes in terms of the intellectual organization of mailboxes. Similarly the ability to save searches, makes it possible to do a good measure of “preprocessing.” In the same way that Gmail changes the email paradigm by saying “don’t think about organizing your email, just do what you need to do,” not much says “do less with your email, don’t organize it, and trust that the machine will be able to help you find what you need when the time comes.”

I’ve been saying variations of the following for years, but I think on some level it hasn’t stuck for me. Given contemporary technology, it doesn’t make sense to organize any kind of information that could conceivably be found with search tools. Notmuch proves that this works, and although I’ve not been able to transfer my personal email over, I’m comfortable asserting that notmuch is a functional approach to email. To be fair, I don’t feel like my current email processing and filtering scheme is that broken, so I’m a bad example.

The questions that this raises, which I don’t have a particularly good answers for, are as follows:

Are there good tools for the “don’t organize when you can search crew,” for non-email data? And I’m not just talking about search engines themselves (as there are a couple: xapian, namazu), or ungainly desktop GUIs (which aren’t without utility,) but the proper command-line tools, emacs interfaces, and web based interfaces?
Are conventional search tools the most expressive way of specifying what we want to find when filtering or looking for data? Are there effective improvements that can be made?
I think there’s intellectual value created by organizing and cataloging information “manually,” and “punting to search” seems like it removes the opportunity to develop good and productive information architectures (if we may be so bold.) Is there a solution that provides the ease of search without giving up the benefits that librarianism brings to information organization?

Gear Mongering

2010-07-27T00:00:00-04:00

At the end of the day (or even the beginning,) I’m just another geek, and despite all of the incredibly (I’d like to think at least) reasoned ways I think about the way I use technology, I occasionally get an old fashioned hankering for something new. We’ve all had them. Perhaps my saving graces are that I do occasionally need new things (computers wear out, cellphones are replaced, needs change), and the fact that I’m both incredibly frugal and task oriented (rational?) about the way I use technology.

But I’m still a geek. And gear is cool. Thoughts, in three parts.

Part One, Phones and the HTC EVO

I’ve been using a Blackberry for 18 months, and I’ve come to two conclusions: blackberries are great and they have the “how ot integrate messaging into a single usable interface.” I was skeptical at first, and it’s very simple, and I’ve never quite gotten my email to function in an ideal way, largely because I think the Blackberry works really well for email. It’s everything else that I might want to do with my phone that I can’t, and I’d probably like to: I have an SSH client and it’s nearly usable. Nearly. I have IM clients, that are nearly functional. Nearly.

When I got the Blackberry, is and was the most important communication I was doing. I worked for a very email-centric company, and I wanted to be able to stay in the email-loop even when I was off doing something else. These days, IRC and XMPP are a far more central feature of my digital existence, and I tend to think that it’s not an Internet connection if I can’t open SSH. I’m also switching on a much longer public-transit focused commute in the next few week, and being able to do research for writing projects will be nice. I’m not sure what the best solution is exactly, though the HTC EVO is a pretty swell phone.

As the kids these days say, “Do want.”

Part two, Infrastructural Computing and Home Servers

I’ve fully adopted an infrastructural approach to technology, at least with regards to my own personal computing. That was a mouthful. Basically, although I work locally on the machine that’s in front of me (writing, email, note taking, collaboration,) much of the “computing” that I interact with isn’t actually connected to the machine I interact with directly. In some ways, I suppose this is what they meant when they said “cloud computing,” but the truth is that my implementation is somewhat more… archaic: I use a lot of SSH, cron, and a little bailing wire to distribute computing tasks far and wide, and the process of moving everything in my digital world from a laptop that I carried around with me everywhere (college,) to a more sane state of affairs has been a long time coming.

Right.

The long story short is that aside from a machine (my old laptop) that’s at capacity powering my “stereo,” I don’t have any computer’s at home aside from my laptop, and I tend to take it everywhere with me, which makes it unideal for some sorts of tasks. Furthermore, without an extra machine setting around, file storage, some kinds of backups, are somewhat more complicated than I’d like. So, I’m thinking about getting some sort of robust server-type machine to stick in a corner in my apartment.

Not exactly sure what the best option is there. I’m burdened by: frugality, sophisticated tastes, and the notion that having quality hardware really does matter.

More thinking required.

Part three, More Laptops

So I might have a laptop related illness. Back in the day, laptops always seemed like a frivolity: underpowered, never as portable as you wanted, awkward to use, and incredibly expensive. Now, laptops are cheap, and even the Atom-based “netbooks,” are functional for nearly every task. I tend to buy used Thinkpad Laptops, and as I think about it, I’ve probably spent as much on the three Thinkpads, all of which are still in service, as I did on any one mac laptop.

The thing about my current laptop is that when you think about it, it’d make a decent home server: the processor has virtualization extensions, the drive is fast (7200 rpm) and it can handle 4 gigs of ram (and maybe more.) What more could I want? And I distributed things correctly, the “server” laptop could be pressed into service as a backup/redundant laptop, in case something unforeseen happened.

Or I could dither about it for another few months, and come to some other, better, fourth solution.

Onward and Upward!

Creating Useful Archives

2010-07-22T00:00:00-04:00

I’ve done a little tweaking to the archives for dialectical futurism recently, including creating a new archive for science fiction and writing and being who I am this has inspired a little of thought regarding the state and use of archives of blogs.

The latest iteration of this blog has avoided the now common practice of having large endless lists of posts organized by publication month or by haphazardly assigned category and tag systems. While these succeed at providing a complete archive of every post written, they don’t add any real value to a blog or website. I’m convinced that one feature of successful blogs moving forward will be archives that are curated and convey additional value beyond the content of the site.

Perhaps blogs as containers for a number of posts will end up being to ephemeral than I’m inclined to think about them, and will therefore not require very much in the way of archives, Perhaps, Google’s index will be sufficient for most people’s uses. Maybe. I remain unconvinced.

Heretofore, I have made archives for tychoish as quasi-boutique pieces: collections of the best posts that address a given topic. This is great from the perspective of thinking about blog posts as a collection of essays, but I’ve started to think that this may be less less useful if we think of blogs as a collection of resources that people might want to have access to beyond it’s initial ephemeral form.

Right now my archives say “see stuff from the past few months, and several choice topics on which I wrote vaguely connected sequences of posts.” The problem with the list of posts from the last few months is that beyond date, there’s not a lot of useful information beyond the title and the date. The problem with the topical archives is that they’re not up to date, their not comprehensive even for recent posts, and there’s little “preview” of a given post beyond it’s title. In the end I think the possibility of visiting a topical archive looking for a specific post and not finding it is pretty large.

In addition to editorial collecting, I think archives, guides, or indexes of a given body of information ought to, provide some sort of comprehensive method for accessing information. There has to be some middle ground.

I think the solution involves a lot of hand mangling of content, templates, and posts. I’m fairly certain that my current publication system is probably not up for the task without a fair amount of mangling and beating. As much as I want to think that this is an problem in search of the right kind of automation, I’m not sure that’s really the case. I’m not opposed to editing things by hand, but it would increase the amount of work in making any given post significantly.

There is, I suspect, no easy solution here.

In Favor of Simple Software

2010-07-13T00:00:00-04:00

I’ve spent a little bit of time addressing some organizational and workflow angst in the past few weeks, and one thing I’d been focusing on had been to update and fine tune my emacs (text editor) and irssi (irc/chat) configuration. Part of my goal had been to use irssi-xmpp to move all of my chat/synchronous communication into one program; unfortunately I’ve not been able to get irssi-xmpp to build and function in a fully stable way. This is probably because I’m hard on software and not because of anything specific to the software itself.

In any case, this lead me to come to the following conclusion about these programs, as they are probably the two most central and most heavily used applications in my arsenal, and without a doubt are the applications that I enjoy using the most. I scribbled the following note a few days ago in preparation for this entry:

In many ways the greatest advance or feature that these programs provide isn’t some killer feature, it’s a simple but more powerful abstraction that allows users to interact with their problem domain. Emacs is basically a text-editing application framework, and provides users with some basic fundamentals for interacting with textual information, and a design that allows users to create text editing modalities or paradigms which bridge the divide between full blown applications and custom configurations. By the same token, Irssi is really a rather simple program that’s easy to script, and contains a number of metaphors that are useful for synchronous communication (chat.)

And we might be able to expand this even further: these are two applications that are not only supremely functional, but are so usable because they are software projects that really only make sense in context of free software.

I want to be very careful here: I don’t want to make the argument that free software isn’t or can’t be commercial, because that’s obviously not the case. At the same time, free software, like these applications needn’t justify itself in terms of “commercial features,” or a particular target market in order to remain viable. It’s not that these programs don’t have features, it is that they have every feature, or the potential for every feature, and are thus hard to comprehend and hard to sell. Even if it only takes a use over a short period of time for users to find them incredibly compelling.

The underlying core extensibility that both of these “programs” have is probably also something that is only likely to happen in the context of open source or free software. This isn’t to suggest that proprietary software doesn’t recognize the power or utility of extensible software, but I don’t think giving users so much control over a given application makes sense from a quality control perspective. Giving users the power to modify their experience of software in an open ended fashion, also gives them the power to break things horribly, and that just doesn’t make sense from a commercial perspective.

There’s probably also some hubris at play: free software applications, primarily these two, are written by hackers, with a target audience of other hackers. Who needs a flexible text editing application framework (e.g. emacs), but other programmers. And the primary users of IRC for the past 8-10 years have largely been hackers and developers and other “geek” types. irssi is very much written for these kinds of users. To a great extent, I think it’s safe to suggest that when hackers write software for themselves, this is what it looks like.

The questions that must linger is: why isn’t other software like this? (Or is it, and I’m missing it in my snobbishness,) and where is the happy medium between writing software for non-hackers and using great software (like these) to “make more hackers.”

Onward and Upward!

Focus and Context Switching

2010-07-08T00:00:00-04:00

This post is inspired by Cory Doctorow’s Inventory about the technology he uses for his work and ongoing personal angst and gadget lust.

For the past six months or so I collapsed my entire computing existence into one, single, computer. It’s a nifty Thinkpad x200, a small laptop with just the right balance of usable screen space, sufficient computing power, and great build quality. And when I say “everything,” I mean it. I hook it up to a monitor and keyboard at the office and do my day-job (technical writing/sys admin stuff) on this system, I have a similar “desktop” situation at home, and I write fiction, do all of my email, write and blog posts, off of this system. I even have development web servers running here.

In a number of ways it’s great, and I wouldn’t trade this for the world. Everything just works the way I want it to, and I never have to worry that I’ve left some important edits to a file on a system elsewhere. Everything is always with me.

Now to be fair, I have additional computers. My old(der) desktop at home keeps backups of files, plays music, and does a number of other tasks. I have (and use) the server that this and other websites run on for some tasks, and I have another instance at the office that manages some work functions, but despite their varying physical distance from me at any given point, my interactions with these computers is always as if they’re remote. When I use a computer, it’s this one.

Now there isn’t a real problem here, except that from a workspace and mindspace perspective the context switching can be somewhat complicated and frustrating. While I’ve got most of the kinks worked out of the docking (monitors and keyboards) process figured out, it takes me a few moments to settle into or out of “laptop-mode” or “workdesk-mode” or “homedesk-mode.” While not having to worry if my files are all up to date, it’s also somewhat distracting for all my different projects to be open all the time. The article I’m working on for work is always open and a few key presses away from the novel I’m working on or the latest in-progress blog post.

Again, this isn’t a really huge issue, but it means that when I get somewhere and want to begin working I have to take a deep breath, and spend a moment or two getting going again. These do fit into the category of “first world problems,” and I’m not sure if there’s a really easy solution. I’ve toyed with a number of resolutions to this angst:

Distribute my existing machines such that I have a machine that lives at work and a machine that lives at home so that I can just sit down at a desk and start working without shenanigans. Given the available hardware, this might mean that I’d spend most of my time using systems that I’m not particularly fond of.
I’ve toyed with having a “tycho writing laptop” that wouldn’t have a web browser installed, for more distraction free writing. I’ve got my old laptop set up to do this, but I’m not likely to take it anywhere (i.e. for the commute,) so it might make sense to get a little netbook for the train for this function, but that seems like overkill.
I might redistribute more of my workload to servers rather than doing everything on the laptop. I’m thinking about having the terminal sessions that I use for email primarily live in screen sessions elsewhere.
I’ve thought about getting a second laptop, (like my main system at the moment,) both for redundancy and to help reduce the cost of switching between various contexts.

As I’ve been toying with this, I’ve been making a number of tweaks to my work flow to help address the pain of context switching, most of which are too trivial and too specific to outline here. Mostly I’ve been tweaking some customizations, improving how I use virtual desktops. While these tweaks have improved things greatly, better internal system management doesn’t solve the underlying issues: it takes time to reconnect to networks, to close tabs in the web browser, to get to the relevant files open in emacs, and navigate to the proper desktop. All the while other contexts (other files, other virtual desktops,) lurk nearby.

And figuring out how to solve this problem involves a certain amount of “head game” for me: avoiding having “the old laptop” be the primary computer for a given task, making sure that I don’t need a network connection for essential tasks. Assorted other weirdnesses.

If anyone recognizes features of this angst that in their own work they’ve managed to resolve, I’d love to hear about your setup.

Fan Fiction is Criticism

2010-07-06T00:00:00-04:00

Thanks to Shaun Duke for inspiring this little rant.

I must confess that I’m mostly uninvolved in the world of fan fiction these days, though I have traveled in “fanish” circles at various points in my past. It’s not because I don’t think fans have interesting things to say about literature and media, or that I don’t think what’s happening in fandom important and fascinating. No, I’m mostly withdrawn because I have too much on my plate and participating in fandom doesn’t really contribute to the specific goals I have at this moment. But I sometimes feel that way about social science.

In any case, I’d like to put forth the following arguments for viewing fan fiction as a form a literary criticism rather than a literary attempt in it’s own right:

Fan fiction is a form of literary criticism. Sure it’s casual, sure it’s written in the forum of a story, but the fan fictioner and the critic both write from the same core interest in interpreting texts and using varying readings of texts to create larger understandings of our world.
The fact that fan fiction looks like a story, is mostly distracting to what’s happening in these texts. Fan fiction, has always been written in communities. The people who read fan fiction are largely the people who write fan fiction. Fan fiction inspires
The quality of fan fiction is also largely irrelevant to the point of whether fan fiction is worthwhile. More so than other forms of writing, fan fiction is less about the technical merits of the text, and more about the discursive process under which the texts are created. Better quality writing makes better fan fiction, certainly but I don’t think fan fiction centers on those kinds of values.
Copyright, and the “intellectual property” status of fan fiction is also sort of moot. It’s true that if we’re being honest fan fiction impinges upon the copyright of the original author. At the same time, fan fiction doesn’t really hurt creators: people aren’t confused that fan fiction is “real fiction,” fan fiction by and large doesn’t divert sales from “real fiction,” and so forth. Sure, it’s a bit weird for some others to find other people playing in their sand boxes, but the truth is that authors have never had a great deal of control over what happens to their work post-publication, so it’s fair.

Additionally, I think that fan fiction accomplishes something that are incredibly powerful and worthwhile that “normal” fiction cannot accomplish. Writing fan fiction can be, I’d wager, an incredibly effective educational experience for new writers, particularly genre fiction writers. By providing a very fast feedback loop with an audience of readers and writers (and lovers of literature and story telling.) Not to mention the fact that because fan fiction tends to be somewhat ephemeral and there’s a wealth of inspiration and impetus for fiction, fan writers can write a lot, and if they choose in a very productive sort of way.

And that is almost certainly a good thing.

The Overhead of Management

2010-07-01T00:00:00-04:00

Every resource, every person, every project, every machine you have to manage comes with an ongoing cost. This is just as true of servers as is it is of people who work on projects that you’re in charge of or have some responsibility for, and while servers and teammates present very different kinds of management challenges, working effectively and managing management costs across contexts is (I would propose) similar. Or at least similar enough to merit some synthetic discussion.

There’s basically only one approach to managing “systems administration costs,” and that’s to avoid it as much as possible. This isn’t to say that sys admins avoid admining, but rather we work very hard to ensure that systems don’t need administration. We write operating systems that administer themselves, we script procedures to automate most tasks as much as possible (the Perl programing language was developed and popularized for use of easing the administration of UNIX systems,) and we use tools manage larger systems more effectively.

People, time, and other resources cannot be so easily automated, and I think in response there are two major approaches (if we can create a somewhat false dichotomy for a moment:)

On the one hand there’s the school of thought that says “admit and assess management costs early, and pay them up front.” This is the corporate model in many ways. Have (layers upon layers of) resources dedicated to managing management costs, and then let this “middle management” make sure that things get done in spite of the management burden. On servers this is spending a lot of time choosing tools, configuring the base system, organizing the file system proactively, and constructing a healthy collection of “best practices.”

By contrast, the other perspective suggests that management costs should only be paid when absolutely necessary. make things, get something working and extant and then if something needs to be managed later, do it then and only as you need. On some level this is inspiring philsophy behind the frequent value of favoring “working code” over “great ideas” in the open source world.¹ Though I think they phrase it differently, this is the basic approach that many hacker-oriented start ups have taken, and it seems to work for them. On the server, this approach is the “get it working,” approach, and these administrators aren’t bothered by having to go in every so often to “redo” how things are configured, and I think on some level this kind of approach to “management overhead” grows out of the agile world and the avoidance of “premature optimizations.”

But like all “somewhat false dichotomies,” there are flaws in the above formulation. Mostly the “late management” camp is able to delay management most effectively by anticipating their future needs (either by smarts or by dumb luck) early and planning around that. And the “early management” camp has to delay some management needs or else you’d be drowned in overhead before you started: and besides, the MBA union isn’t that strong.

We might even cast the “early management” approach as being “top down,” and the “late management” camp as being “bottom up.” If you know, we were into that kind of thing. It’s always, particularly in the contemporary moment to look at the bottom-up approach and say “that’s really innovative and awesome, that’s better,” and view “top-down” organizations as “stodgy and old world,” when neither does a very good job of explaining what’s going on and there isn’t inherent radicalism or stodginess in either organization. But it is interesting. At least mildly.

Thoughts? Onward and Upward!

Alan Cox’s Cathedrals, Bazaars and the Town Council
↩

Jekyll Publishing

2010-06-29T00:00:00-04:00

I wrote about my efforts to automate my publishing workflow a couple of weeks ago, (egad!) and I wanted to follow that up with a somewhat more useful elucidation of how all of the gears work around here.

At first I had this horrible scheme setup that dependent on regular builds triggered by cron, which is a functional, if inelegant solution. There’s a lot of tasks that you can give the appearance of “real time,” responsiveness by scheduling more brute tasks regularly enough. The truth is, however, that its not quite the same, and I knew that there was a better way.

Basically the “right way” to solve this problem is to use the “hooks” provided by the git repositories that I use to store the source of the website. Hooks, in this context refer to a number of scripts which are optionally run before or after various operations in the repositories that allow you to attach actions to the operations you perform on your git repositories. In effect, you can say “when I git push do these other things” or “before I git commit check for these conditions, and if they’re not met, reject the commit” and so forth. The possibilities can be a bit staggering.

In this case what happen is: I commit to the tychoish.com repositories a script that synchronizes the appropriate local packages runs and publishes changes to the server. It then sends me an xmpp message saying that this operation is in progress. This runs as the post-commit hook, and for smaller sites could simply be ”git push origin master”. Because tychoish is a large site, and I don’t want to be rebuilding it constantly, I do the following:

 #!/bin/bash 

 # This script is meant to be run in a cron job to perform a rebuilding
 # of the slim contents of a jekyll site. 
 #
 # This script can be run several times an hour to greatly simplify the
 # publishing routine of a jekyll site.

 cd ~/sites/tychoish.com/

 # Saving and Fetching Remote Updates from tychoish.com
 git pull >/dev/null && 

 # Local Adding and Committing
 git checkout master >/dev/null 2>&1
 git add .
 git commit -a -q -m "$HOSTNAME: changes prior to an  slim rebuild"  >/dev/null 2>

 # Local "full-build" Branch Mangling
 git checkout full-build >/dev/null 2>&1 &&
 git merge master && 

 # Local "slim-bild" Branch Magling and Publishing
 git checkout slim-build >/dev/null 2>&1 &&
 git merge master && 
 git checkout master >/dev/null 2>&1 &
 git push --all

 # echo done

Then on the server, once the copy of the repo on the server is current with the changes published to it (i.e. the post-update hook), the following code is run:

 #!/bin/bash
 #
 # An example hook script to prepare a packed repository for use over
 # dumb transports.
 #
 # To enable this hook, make this file executable by "chmod +x post-update".

 unset GIT_DIR
 unset GIT_WORKING_TREE

 export GIT_DIR
 export GIT_WORKING_TREE

 cd /path/to/build/tychoish.com
 git pull origin; 

 /path/to/scripts/jekyll-rebuild-tychoish-auto-slim & 

 exit

When the post-update hook runs, in runs in the context of the repository that you just pushed to, and unless you do the magic (technical term, it seems) the GIT_DIR and GIT_WORKING_TREE variables are stuck in the environment and the commands you run fail. So basically this is a fancy git pull, in a third repository (the one that the site is built from.) The script jekyll-rebuild-tychoish-auto-slim looks like this:

 #!/bin/bash
 # to be run on the server

 # setting the variables
 SRCDIR=/path/to/build/tychoish.com/ 
 DSTDIR=/path/to/public/tychoish/
 SITENAME=tychoish
 BUILDTYPE=slim
 DEFAULTBUILD=slim

 build-site(){
      cd ${SRCDIR}
      git checkout ${BUILDTYPE}-build >/dev/null 2>&1
      git pull source >/dev/null 2>&1
      
      /var/lib/gems/1.8/bin/jekyll ${SRCDIR} ${DSTDIR} >/dev/null 2>&1
      echo \<jekyll\> completed \*${BUILDTYPE}\* build of ${SITENAME} | xmppipe garen@tychoish.com

      git checkout ${DEFAULTBUILD}-build >/dev/null 2>&1
 }

 build-site;

This sends me an xmpp message when the build has completed. And does the needful site rebuilding. The xmppipe command I use is really the following script:

 #!/usr/bin/perl
 # pipes standard in to an xmpp message, sent to the JIDs on the commandline
 #
 # usage: bash$ `echo "message body" | xmppipe garen@tychoish.com
 #
 # code shamelessly stolen from:
 # http://stackoverflow.com/questions/170503/commandline-jabber-client/170564#170564

 use strict;
 use warnings;

 use Net::Jabber qw(Client);

 my $server = "tychoish.com";
 my $port = "5222";
 my $username = "bot";
 my $password = "";
 my $resource = "xmppipe";
 my @recipients = @ARGV;

 my $clnt = new Net::Jabber::Client;

 my $status = $clnt->Connect(hostname=>$server, port=>$port);

 if (!defined($status)) {
    die "Jabber connect error ($!)\n";
 }
 my @result = $clnt->AuthSend(username=>$username,
    password=>$password,
    resource=>$resource);

 if ($result[0] ne "ok") {
    die "Jabber auth error: @result\n";
 }

 my $body = '';
 while (<STDIN>) {
    $body .= $_;
 }
 chomp($body);

 foreach my $to (@recipients) {
     $clnt->MessageSend(to=>$to,
             subject=>"",
             body=>$body,
             type=>"chat",
             priority=>10);
 }

 $clnt->Disconnect();

Mark the above as executable and put it in your path somewhere. You’ll want to install the Net::Jabber Perl module, if you haven’t already.

The one final note. If you’re using a tool like gitosis to manage your git repositories, all of the hooks will be executed by the gitosis user. This means that this user will need to have write access the “build” copy of the repository and the public directory as well. You may be able to finesse this with the +s “switch uid” bit, or some clever use of the gitosis user group.

The End.

The Meaning of Work

2010-06-22T00:00:00-04:00

I’ve started to realize that, fundamentally, the questions I’m asking of the world and that I’m trying to address by learning more about technology, center on work and the meaning and process of working. Work lies at the intersection of the all the things that I seem to revisit endlessly: interfaces, collaboration technology, cooperatives and economics institutions, and open source software development. I’m not sure if I’m interested in work because it’s the unifying theme of a bunch of different interests, or this is the base from which other interests spring.

I realize that this makes me an incredibly weird geek.

I was talking to caroline about our respective work environments, specifically about how we (and our coworkers) relocated (or didn’t) for our jobs, and I was chagrined to realize that this novel that I’ve been working at (or not,) for way too long at this point spends some time revolving around these questions:

How does being stuck in a single place and time constrain one agency to effect the world around them?
What does labor look like in a mostly/quasi post-scarcity world?

Perhaps the most worrying thing about this project is that I started writing this story in late August of 2008. This was of course before the American/Financial Services economic crash that got me blogging and really thinking about issues outside of technology.

It’s interesting, and perhaps outside the scope of this post, but I think it’s interesting how since graduating from college, my “research” interests as they were, all work them into fiction (intentionally or otherwise.) I suppose I haven’t written fiction about Free Software/open source, exactly, but I think there’s a good enough reason for that.¹

I’m left with two realizations. First, that this novel has been sitting on my plate for far too long, and there’s no reason why I can’t write the last 10/20 thousand words in the next few months and be done with the sucker. Second, I’m interested in thinking about how “being an academic” (or not) affects the way I (we?) approach learning more about the world and the process/rigor that I bring to those projects.

But we’ll get to that later, I have writing to do.

I write fiction as open source, in a lot of ways, so it doesn’t seem too important to put it in the story as well.
↩

There's Always Something New to Learn

2010-06-17T00:00:00-04:00

Now that I’m fairly confident in my ability to do basic Linux systems administration tasks: manage web and email servers, maintain most Linux systems, convince desktop systems that they really do want to work the way they’re supposed to, I’m embarking on a new learning process. I’ve been playing around with “real” virtualization on my desktop, and I’ve been reading a bunch about systems administration topics that are generally beyond the scope of what I’ve dealt with until now. Here is a selection of the projects I’m playing with:

Getting a working xen setup at home. This requires, learning a bit more about building working operating systems, and also in the not to distant future buying a new (server) computer.
Installing xen on the laptop, because it’ll support it, I have the resources to make it go, and it’ll be awesome.
Learning everything I can about LVM, which is a new (to me) way of managing partitions and disk images, that makes backups, disk snapshots, and other awesomeness much easier. It means, some system migration stuff that I have yet to tinker with, as none of my systems currently support LVM.
Doing package development for Arch Linux, because I think that’s probably within the scope of my ability, because I think it would add to my skill set, and because I appreciate the community, and I want to be able to give back. Also I should spend some time editing the wiki, because I’m really lazy with that.

I guess the overriding lesson of all these projects is a more firm grasp of how incredibly awesome, powerful, and frankly stable Arch Linux is (or can be.) I mean there are flaws of course, but given how I use systems, I’ve yet to run into something show stopping. That’s pretty cool.

The Old Projects Project

2010-06-16T00:00:00-04:00

Before a road trip a, by now, a couple of months ago, I installed a copy of nginx on my laptop on the hope of doing some web development and working on other projects when I was in the car. For the uninitiated (you mean you don’t all write technical documentation for web developers and systems administrators?!?) nginx is an incredibly powerful web server. As of June 11th, foucualt the server that hosts the Cyborg Institute and tychoish.

This is, almost always, I think, a loosing proposition.

I never get any sort of substantial (or insubstantial) work done during my road trips up-and-down the north east corridor. Not that that’s a bad thing, but I also expect that there’ll be more awake-time when I’m not driving or gossiping.

And there never is.

So the web server sat unused for a long time on my laptop, but recently I’ve been playing with it a bit and I’ve finally gotten a number of cool things set up. I have a local “git web” instance which makes it easier to track progress on local and private projects that are stored in git. Perhaps more importantly, I have set up quick local ikiwiki instances for a number of projects. They’re easy to configure, quick to setup, and while I suppose I could hack something together in nifty for myself, there’s something nifty about being able to take an alternate view of some content and also being able to really preview changes to you work before publishing them.

Also, and the real reason for this post, is that by virtue of this development, I have revisited a few projects that had been lingering in the home directory of my computer for far too long. Which has been a powerful and useful exercise.

By which I mean, it’s been painful.

Besides “the novel,” which has been the lingering and dragging front burner project for a year, there are a number of quasi-serial stories that have lingered in some state of incompleteness for a couple of years now. I’m kind of amazed both at how foreign these stories seem to me both in terms of the style (good to know that I’m a better writer than I was a few years ago,) and also how quickly I can fall right back into the story and tell you every little thing about the world, situation, and moment where I left off.

The mind is, indeed, an amazing thing.

Where my strategy for the past year has been to “plow through and finish the novel,” I think my tactic this summer will be to move all of my projects forward in some way. Small daily writing goals for the novel, combined with somewhat less regular (but more specific) goals with regards to other projects. In the next two months I want to have a fairly active and varied writing schedule worked out that isn’t based around the monthly (or so) weekend binges that I’ve been using for most of the last year.

That’s the plan at any rate.