I have this algorithm...

Algorithms can solve two kinds of problems.

1) Problems with verifiable answers and...
2) Those without verifiable answers (or the verification is proved by statistically sampling)

Verifiable answers:

Algorithms in this category address problems like, find me all the numbers in this list, or even, find me the fastest route between point a and b.

The value in the algorithm is not the correctness, it is easy to find the answers to these problems, rather the value in these algorithms are the speed with which they are executed and the amount of data you have to solve your problem.

So for example there are many different kinds of sorting algorithms - they all correctly sort, but some are faster and some use less memory. This is where the ingenuity or value of the sorting algorithm comes in.

For the mapping example, it is not difficult to map the fastest route between two points. But you could imagine an algorithm that used realtime traffic data would provide faster routes than other algorithms. The answer these algorithms may be different, but their actual processing of information - the algorithm itself - may be the same. The only difference is the input data used by the algorithm.


Now onto the algorithms with no answers -
these are the interesting algorithms that answer questions like - find the top 10 sites on music, or recommend movies I'll like, or find me a date.

In the case of search - the answer can be the top 10 sites on music are the top music sites that people link to (google's algorithm), or the top 10 sites that my friends like (a facebook solution)

In the case of recommendations, we would have an algorithm like that produced by the netflix prize, a statistical sampling of user preferences run through a barrages of matrix filters. We could also have the pandora algorithm, create a genetic code for a piece of music based on frequency, btm, etc and then find other similar genetic codes.

For dating there is a little less innovation out there. Most sites ask a set of questions (perhaps developed by behavioral specialists), and match those with similar or compatible answers, or let users shop for dates by displaying the answers to these behavioral questions.

But that is it go forth and create algorithms for problems with no answers

transducer

i feel like on of the big new media memes is transduction.

For example convert stock market feeds into audio sounds, or convert twitter feeds into graphical visualization.

Data visualization and audio visualization are forms of transduction - translating from one form into another. The issue with transduction is how do we perform the translation. This is where the real art of data visualization or data sonification takes place. There are infinate numbers of ways to map the frequency of #brit on twitter to a visual cue. What signifies frequency (line size?), what signifies subject matter (color?), where do you place the visualization on the screen (existence in different social networks?). All of these are arbitrary and the criterion of success is how transparent the visualization is for understanding the data, or how aesthetically pleasing the result is.

I have been writing an application called doh rae tweet which lets you compose music via twitter. Here again the mappings are arbitrary. an 'A' is a note 'a', but a space is S (it could just be a space). The default time signature is 4/4 unless you start your tweet with something else like 8|8. and you adjust the beat of a note by + or - so A- (in 4/4) would be a half note. This is all arbitrary and it is not necessarily transduction but translation. Not transforming one type of thing into something else, but representing something (musical notation) in something else (140 characters on twitter) . In this way it is much more similar to writing a compiler than writing a data visualization.

However doh rae tweet can also can be used as a transduction engine. I can feed through random twitter feed and use the music compilation engine to hear what that tweet sounds like according to my 'compiler' Although it probably wont sound very nice. Most letters will not have an audio mapping. So for this sort of application I would want to replace my compiler or lexical engine with another. Ideally anyone could create their own mapping. I think it would be interesting to do this via a GUI and feedback loop so you could adjust your mappings visually as you hear the effects of your changes.

One final though - I also want to use this mapping for other notation schemes. One that seems a natural is chess. With the 8x8 chess board mapping to the diatonic scale. However, this too would need its own mapping. ( Also what do you play, to just play the chess piece that is moved - or all the chess pieces. ) I went online to see if anyone has attempted sonficiations of chess notation and found the following:

Halfbakery This has some links relating chess to music
Chess History A list of articles and historical documents connecting chess with music (not very informative but interesting from a historical standpoint).
Music and Chess A great website with all sort of information linking chess and music. I got this from that site: "The Oxford Companion to Chess is a comprehensive encyclopedia of chess. It contains articles on history, terminology, chess players, and the relationship between chess and other subjects such as music, art, theatre, literature and philosophy. Many of the terms listed in this book are also musical terms. For example, in chess, a person who creates puzzles and problems to be solved is called a composer, and two different sequences of moves that lead from one given position to another are said to be related by transposition. Some other terms that are used in chess and music are: play, piece, notation, score, tempo, theme, variation, development, minimal composition, round, major and minor, position, second, retrograde, mirror, attack, anticipation, phase and echo."

Blown away by The Sound and the Fury

Ok - so when I first started the sound and the fury I had no idea what was going on. 'I think they are on a golf course' I told my business partner Paul. 'I have no idea what is going on.' Half way through the book I felt that I was waisting precious moments of my life reading a piece of crap. However, at some point - probably during Quentin's chapter - I reevaluated.

Why did I reevaluate?? I dont know. Perhaps the language... there is a certain lyricism - and i am a sucker for dialogue. But, at the end of the book a few things struck me...

1) Such a wide expanse of time was covered in such a 200 page book. The fragmented storytelling conveyed more information and feeling than a 1000 page tomb of today's magical realism.
2) What is the story?? The book is really about the construction of narrative, or the construction of myth and meaning. At the end of the day - when you read a story or a myth everything seems so tidy - but the construction itself is messy. While the story is being written you have no sense of the whole completed version - the sound of the fury is like a story writing itself
3) Roshimon - Whose point of view is correct? Seeing the compson family through the eyes of the different narrators in a stream of consciousness voice bring home the fact that every story is told from a point of view.
4) It is a mind fuck - the story seems like a puzzle. Not a post modern puzzle, where the whole point of the novel is the puzzle (like paul auster - of whom I am a big fan). Perhaps puzzle is the wrong word... It is coded. I feel like if i diagramed the whole novel out - parsing out the different stories by the different narrators at different times in the life of the characters I would get another story. There is a second story hidden in the story.
5) This is a modern novel. It struggles with modern themes (and it was written in 1930s I think). Sexuality, Economics, Race, Class - the treatment of these issues are not preachy but are integral to the lives of the character. this is great storytelling - bringing up weighty issues as part of the characters struggles

Anyway I am inspired - I may experiment with this sort of writing for some of my short stories. Also reading Sarte's critique of faulkner - I'm into sarte as literary critic (just was gifted book 5 of the family idiot about flaubert).

Local build of ffmpeg

This was going to be a highminded post about fairy tales and morphology but instead i am going to record my trials and trivails of running ffmpeg on amazon's map-reduce.

So - what i am doing is adding a cached directory of ffmpeg to every map-reduce computer i use


#!/usr/bin/bash
./elastic-mapreduce --create --stream --alive --name $1 \
-c credentials.json \
--input s3n://13bit.videoout \
--mapper s3://13bit.scripts/compression.py \
--reduce s3://13bit.scripts/reduce.py \
--output s3n://13bit.videoout.ts \
--log-uri s3n://13bit.log \
--cache-archive s3n://13bit.scripts/ffmpeg.zip#ffmpeg

in order to build ffmpeg i started up a debian sqeeze box via aleatic ami (ami-daf615b3) http://alestic.com/. I then had to update apt-get (apt-get update), install git, download ffmpeg and related libraries and run the ffmpeg configure switch --prefix to set the install directory. I am now zipping it up and testing it on another instance of the squeeze debian - at which point i will start up the map reduce process again

i am also looking at boston terrier art on etsy (thanks to Jane Kim)
http://www.etsy.com/listing/37552859/boston-terrier-dog-art-original-painting
I am doing this while my boston terrier sleeps and farts next to me.

Living the Dream

Last night Paul and I attended a fascinating lecture on the postal history of the Congo Free State at the NY philatelic society. This is research for our latest documentary. There is nothing like listening to a lecture by an expert. Not all fields are capable of producing experts - Literary theory for instance. You can only be an expert is something that exists, that has a counterpoint in reality. You cannot be an expert in unicorns, however you can be an expert on the literature of unicorns, the archaeology of unicorns, and the symbology of unicorns. Likewise you cannot be an expert on literary theory, but you can be an expert on the thoughts of literary theorists. However I think expertise in something like stamps of the Congo is much more worthwhile...

On another note- I ran across the following art project by cory arcangle - it culls the internet for blog posts that begin with i'm sorry. I am a big fan of his video work, especially mario clouds video & the the one where he just projects fields of color from the projector. However, this 'sorry' project is sort of in bad taste. What is it? A project mocking people attempting to maintain blog? I am not quite sure what its purpose is. I doubt it is to be inspired by the heart felt apologies - as cory writes on his blog. So what exactly is the point of this project? to call attention to the dead blogs in the world, to show how apologies have become meaningless, is it supposed to just be funny, it it supposed to show how pathetic people are, how people have no imagination (except for cory)? not sure - here is a blog art project i just blew out of my ass - probably similar to the process of other net artists (although my process does not involve weed) - a dictionary of common words as spelled by bloggers ( or twitters)....

More Adventures with Map Reduce and the Cloud

I am compressing video in the cloud via amazon's EC2/mapreduce/s3/etc... I am using map/reduce for this although it is really not a reduce problem only a map problem and not even that - I just need to spawn a bunch of ffmpeg processes....

The problem is I cant map/reduce on a disk that has ffmpeg. So what I did was compiled map reduce on my machine - zipped it - uploaded it to s3 then added a bootstrap action to unzip and ffmpeg whenever the program starts... lets see how this works - I think I need to recompile on debian though - currently using fedora

As for the arabian nights - i have learned that the arabic oral tradition no longer exists because people sit around in cafes and watch tv instead of telling/listening to stories - ahh technology

Deep thoughts on magical realism

So I am continuing my romp through Irwin's commentary to the Arabian Nights - although I really want to read a copy of The Ocean of the Rivers of Story- which apparently is the source of all stories. But I digress.

In my romp I found that fairy tales were considered part of the genre of the Arabian Nights - and were very much in fashion in 1600s Europe. The author speculates that these were stories were relegated to children's tales as tales of the fantastic were usurped by science fiction.

Today I suppose tales of the fantastic are usurped by magical realism. It is interesting to think though that much of literature is to satisfy this desire for surprise and wonder. We are normally taught in school that fiction imparts some sort of cultural norms - such as homeric heroism. Rather literature engages our desire for something beyond the real. Literature is metaphysics. Perhaps these days video games have replaced this fantabulism because even magical realist books are not quite fantastic enough. We definitely see strands of this in contemporary tv such as Lost and true blood. Sometimes, however, a simple rendering of the real, in all its absurdity, is fantastic (hence my continued obsession with tao lin).

Genealogy of manuscripts

I am reading Robert Irwin's "The Arabian Nights" A companion. It was recommended to me as 'the best commentary ever.' I dont really agree. I think the best work of commentary is Herbert Dreyfus's commentary on Heidegger's "Being and Time", "Being-in-the-world." The best work of literary criticism or commentary is Mimesis by Auerbach. I keep a copy of this above my desk - 'being-in-the-world' is relegated to my stacks.

What is interesting about the Arabian nights commentary is its discussion of literary scholarship. This all but disappeared with the advent of ideologies and literary criticism. Rather than the try piecing together of texts from various sources, trying to find the 'true text' in some sort of pseudo-scientific exploration, people now just interpret a text in light of some agenda - in some meta-pseudo-scientific pontification. We are really moving beyond the scientific method - it seems - as a mode of exploration and investigation of the world. String theory is not a result of empirical evidence and nor is the Lacanian interpretation of Pär Lagerkvist.

But back to the Arabian nights commentary. So one of the ideas that caught my eye was that scholars where attempting to create a sort of evolutionary tree of different editions of the Arabian nights. This is sort of an interesting thought to me that works of literature can have a history. Further along, we see that many of the stories in the Nights, have parallels in Chaucer, Boccaccio, various sanskrit works, and other. Here is yet another genealogy - the genealogy of an idea rather than the genealogy of a work. I feel like these sorts of ideas are used in the generation of books themselves, the questionable source of a work is used as a device in Don Quixote and in the "Dictionary of the Khazars" - and probably in 59% of Bevery Borges novels.

The sort of scholarship that went into this sort of detective work seems more apt for the computer age (or the Big Data Age) than the experimental age, however I dont think literature scholars are doing this sort of work anymore -

I leave you with some of my favorite lines from Tao Lin's "Shoplifting from American Apparel"
"On Christmas Eve Sam work around 7 p.m. in his brother's studio apartment in Manhattan. Sam had moved in November into a four-person apartment in Brooklyn but was staying at his brother's studio while his brother was on vacation with his girlfriend. Sam put on music very loud and showered in the dark with the bathroom door open. He put in earphones and walked ten blocks to an organic raw vegan restaurant. He ate a seaweed salad. He drank a smoothie. He walked back to the apartment. He drank an energy drink. He worked on writing for two and a half hours. He lay on his brother's queen-size bed and listened to music. He read most of the newest Stephen Dixon novel and fell asleep around 3 a.m.

The Greatest Work of Our Time Is Lost

I was just watching the imaginarium of dr parnassus which begins full of promise and then sort of craps out. The most interesting part of the film is the story of parnassus not the imaginary wonderlands of random people who go through parnassus' mirror.

I was totally blown away by the production design of Parnassus' Monastery. After watching the DVD extras about the Monastery, it seemed that one of the great achievements of the film were the props. I often think this is the case with other films, and with theater in particular.

It seems after thousands of years of architectural masterpieces, 500 years of scupltural masterpieces, and about 200 years of painted canvas masterpieces - we are in an age where masterpieces are really transitory and in the service of other even more transitory masterpieces. I guess this is what happens when production is in service of entertainment or the triumph of bread on circuses.

A completely different blog post

i have been blogging lately to keep track of what I am doing with my streaming video project. The montagenik blog is probably not the best place for this. I should probably keep this info in evernote or perhaps another blog, or perhaps a del.icio.us account, or perhaps a git gist. However, the fact is, that I actually go back in my blog and look for these tidbits. My blog is like my personal search engine repository. When I spread out this information over a bunch of different application I generally dont use them. I find it more efficient to keep everything in one location.

This week has been a good week in the 13Bit getting things done world, but a bad week in the Meredith getting things done world. The jet lag has put me on a really messed up schedule and I am completely useless after 4pm. Tomorrow Paul and I have a short shooting trip to NJ where we will interview two collectors, then sunday Cindy and I will bot in preparation for the upcoming robot parade. Paul and I also have something special up our sleeves for the Ecopocolypse website. It is very cool....

I started out the day by reading from 'The Book of Disquiet' - This is a fantastic, beautiful, inspiring book. "And at this table in my absurd room, I, a pathetic and anonymous office clerk, write words as if they were the soul'd salvation and I gild myself with the impossible sunset of high and vast hills in the distance." I am considering ripping off Mattins and reading parts of this on a daily podcast.

My new love is the OED on the computer.

More EC2, FFMPEG and video streaming from the cloud

So today I am setting up the compression engine on EC2 to compress video out into 4 formats. I need to do this before I use the segmenter to divide up video out into segments that will then be recompiled on the fly via lexical searches...

Anyway I had a java program downloading my mov files - clips from video out- but it was taking to long. I dont want to have to download the file in order to convert it I want ffmpeg to convert it via http... luckily i found this on stack overload:
http://stackoverflow.com/questions/2677317/how-to-read-remote-video-on-amazon-s3-using-ffmpeg


evermind, I found an easy way to solve my problem.

I set up an amazon cloudfront download distribution pointing to my S3 bucket.
Via cloudfront the files are accessible with ffmpeg over http:

ffmpeg -i "http://subdomain.cloudfront.net/video.mp4" -ss 00:00:10 -vframes 1 -f image2 "image%03d.jpg"

Incoherent Ramblings on EC2 and ICT4D

When you are deploying some sort of web based app with some sort of services (mobile, messaging, whatever) in a far away land, one of the largest time sucks is configuring the server. If the server has already been configured, there is generally no documentation. No one knows you decided to config your ssh port on 666. Granted one 'could' provide better documentation, as well as server install scripts (think a list of apt-get or yum or whatever you are doing). However, after working with EC2 on amazon, I think standardized disk images are the way to go.

1) You are actually deploying the same server everywhere. So, why not make that server once, as an EC2 instance and then actually deploy it.
2) Services that rely on a hardware device such as an SMS modem have to be installed on a local box, but, web access, database storage, and number crunching, can all be stored in the cloud. Then, if you want to edit your web interface or Z-score calculation for child malnutrition, you don't have to worry about power outages in Tanzania. And you will probably have less latency as well. The route that your server takes to ssh into a server in Tanzania is Byzantine.
3) Scaling. This is a dream issue but say all of South Africa starts using your service, all you need to do is deploy some more EC2 instances and load balance. You dont have to send over a bunch of techies to set up a server farm.
4) CAVEAT - as i mentioned above if you have a device attached to your box - such as an SMS modem, or rf transceiver or whatever - then you cannot deploy that aspect of your app to the cloud (the aspect that connects with the device) however you can use the EC2 image to create your local server.

EC2 is great for standardization, especially when you lack people, money, and equipment - i love it.

Superbitter AMI

I am creating an instance for the SuperBitter off the RAILS fedora AMI. I dont use fedora but I decided to use an Amazon recommended AMI and it was fedora or bust.

I added: git, java, csound, ffmpeg, scala, jetty & mavin (for lift)

ffmpeg is still missing some stuff

used this resource for ffmpeg install :http://blog.ryanparman.com/2008/06/28/installing-ffmpeg-php-fedora-amazon-aws/

in the midst of building thrift - (need to update autoconfig), mongodb (sort of a pain), open source segmenter (a pain), http://github.com/mfchang/HTTP-Live-Video-Stream-Segmenter-and-Distributor

For the future I am going to split up the web EC2 (rails) from the backend EC2 (scala, mongodb, csound). But for now I dont want to deal.

Video Server Development

So this is basically to help me keep track of development for a video application I am developing.
Paul sent me the following link that is super useful:
http://www.ioncannon.net/programming/1015/ipad-streaming-video-and-more/
This is also a useful link: http://blog.kyri0s.org/post/271121944/deploying-apples-http-live-streaming-in-a-gnu-linux

The plan:
I am using http live streaming to stream the video. I am streaming 90 minutes of video.

I need to create different bitstreams so I am going to use FFMpeg.

Then I am going to use an open source segmenter instead of the apple segmenter (this is running on unix) http://svn.assembla.com/svn/legend/segmenter/

I am going to set all this up as an instance on EC2.

This means I need to set up an AMI instancehttp://docs.amazonwebservices.com/AmazonEC2/dg/2006-06-26/creating-an-ami.html.

I may even install csound on the AMI

Reading Marx is Making Me Depressed

I feel like I have been reading lots of Marxist inspired thought lately, and so I picked up 'A very short intro to Marx'. Thus started my descent into economic depression. To summarize the summary... history is moving toward the freedom of mankind. Freedom, for Marx, means, freedom from alienation. Human beings are alienated from themselves because their work is not an expression of their nature but a means to survive. Alienation - the key point here. Humans are alienated because of economic conditions. That is what separates us from one another, (according to Marx), is economic conditions. This is opposed to religion (Feuerbach) as the crux of alienation. Or our separation from our true selves (Hegel).

Anyway - I find Marx's critique of alienation spot on - even if his economic theories are a bit dodgy. Everything these days is commodified and monetized. Here, I am blogging for the pleasure of it. However, blogger & google are commodifying this blog by integrating it into their ad/knowledge network. This is even more profound than the realized alienation of the wage slave who must sell his time in order to support himself. In this case I am not even aware of my alienation. I view myself as human, while google views me as a commodity.

Also, as a former wage slave to a corporation, but now an independent wage slave to myself, I doubt there is any way to escape the transformation of time into money. In my thought though the greatest achievements of technology are medical and, in a round about way, social. By social, I mean, technology and technological accomplishment is the great leveler. It is not subject to discrimination. The ability and the actual achievement of technological accomplishment by people of all races, gender, sexual orientation, leads to the end of social discrimination.

I started blogging yesterday and then did not finish

I dont even remember what I was blogging about. But let me free associate here. I am currently reading John Crowley's book "Little, Big." This is a magical realism tale about a new england family that is some how connected with fairies. It is not necessarily plot driven - rather the plot is figuring out what the twist is or why we should be interested in this family in the first place. The book has fairy tale logic.

I have a love/hate relationship with John Crowley's books. I read his Aegypt series on the Kindle when I lived in Hong Kong - and I remember sitting in the IFC court yard during my lunch breaks reading this book and feeling connected to home. I like his ideas. The integration of mystical and magical elements with everyday life. The possibility that esoteric knowledge is practical rather than fantastical. This is not like generic magical realism in that Crowley is integrating historical esoteric elements into his book. He creates a world where the esoteric knowledge, or mystery schools of thought, actually are valid. However, the novels are more like an exploration of these topics rather than a structured story. It is almost like when you are playing a video game and the exploration of the levels is more interesting than the boss fight.

Journey to the west is a tale that treats the 'boss fights' with drama and exposition - fantastic stuff. I like the exploration though, why is probably why I am drawn to Crowley's novels.

Has it really been 3 weeks since my last post!

I wish I had something to write. I have blogger block tho. I blame it on the intense heat that is making me lethargic, unable to focus, and physically weak. There is something about the heat that makes reality less real. So, it is hard to get worked up about work. The flip side of this lethargy is that I am no longer suffering from a low level malaise. How can you feel malaise if it is hard to feel anything. I need inspiration. I was inspired by the 13Bit road trip to document collectors, however I was to exhausted to turn this inspiration into anything concrete. All I can do now is execute the little things, doing laundry, logging footage, reading a poem, walking the dog, and perhaps a short blog post.

the scrivener

I use scrivener to write. Paul and I use it to write our screen plays. I use it to write my stories, plays, and currently I'm using it to write a graphic novel. I was once in an airport bookstore and spied Bartleby the Scrivener, and read it on the flight back to NY. How is scrivener the software like Bartleby the scrivener? Perhaps some industrious 11th grade English student somewhere can deconstruct this. Really I like the scrivener software because I have different outline modes, I can write in a corkboard mode, and it is easy to keep track of research and themes related to a particular topic.

So, can I write a game using Scrivener? Good question. I don't know, maybe I should try that.
I'll post here the first paragraph of my latest short story - it is the documentation for a cellular automata program.

"Documentation for a Computer Program"

Out of the IOStream comes theWorld. The stream is blue, not a microsoft blue, but deep sky blue, #00BFFF, and it is not web safe. theWorld is created from a pseudo random number generator. The first iteration of theWorld was created with a Perlin noise filter. But that produced too much geographic diversity. The purpose of this experiment is to see how cells interact, so we need to strip out all extraneous variables - geographic diversity was the first to go. The pseudo random number generator, and that provides much less variation in worlds from one Universe to the next.

The Great American Novel

My husband always says that I should write a novel. I have no idea where he gets the idea that I am capable of such a feat - but it is an interesting idea to entertain. I have about 20 half finished novels and some new novel ideas, but so does everyone.

A novel is more than an idea. It is not a knock knock joke, a one liner. A novel is not just a good idea - it is the unfolding and refolding and crumpling up and tossing on the floor of everything that surrounds that idea. What novel should I write? I ask my husband. Write that novel about Detroit. I had a idea about a novel that follows generations of a detroit family through immigration, the automotive revolution, and finally to current decay. "How should I start?" I ask. "Dont try to write War and Peace." My husband replies. That is actually very good advice. But the Detroit novel really does seem to have a War and Peace scale, so I try and pick up another idea. Ok - I was thinking about something golem related. For some reason I have been thinking about Golem stories. I have a nascent screenplay on a Golem and World War 2. I also love Jewish esoteric stories (the hasidic tales, tales of nachman) - I like weird esoteric stories in general. So I was thinking about combining the two. A group of jews hiding one night from the pogroms in early 19c russia stay up all night telling stories. Shamelessly stealing from the Decameron, The Canterbury Tales, and other stories in a story. I love framing devices almost as much as the stories they frame. I started writing this but all I got was a frame. A fictional story of my grandmother writing down her recollections from this night 50 years later. And my possession of this manuscript after her death, via my aunt. I actually had a beginning and an ending and all i need to do was fill it with stories. This has proven more difficult that I first thought. (I was also inspired by The Manuscript of Saragasso)

I was going to do 10 stories based on the 10 Sephirot (levels of creation) based on Kabalistic ideas. The first story I was writing was based on Daat (knowledge) and was a fictional account of Spinoza - really a glass grinder in Amsterdam. I was playing with ideas of rationality, speculation. But that is all I got - no plot ! I am thinking that there should be a video game based on the 10 sephirot. Maybe I should do that instead.

Being stymied, I decided to get into shape by writing short stories. Short stories are a different beast than novels - and I think most novelist dont get this. At the moment I am enjoying the short stories of George Saunders - he writes some dystopic fiction about amusement parks or modes of entertainment. Really great stuff. Since I was reading about vagueness in philosophy - I thought it would be interesting to write a story that explores vagueness. I sketched out a story about a bunch of adolescent girls who go to an amusement park and the different rides they can take based on the 'You are this high' mark - a sort of exploration of living in vague territory. I was going to end it with a death - but that was really a bummer. I really could not get behind it. It did not seem like a story I would write. And it really did not have a 'turn', a strong idea. A short story is sort of like a knock knock joke. Maybe I should write some science fiction.

This post was written while listening to passion pit.

Know your audience

I was in Los Angeles last week, and stayed with two friend, Arv and Anjali. Arv is an economist working on developmental economics - so we have lots to discuss - especially the fact that after recent experiences I no longer believe non-profit work is a viable solution to address public health and poverty issues. I have become a die hard capitalist - and believe the only solutions that address issues of poverty and health crises must be self-sustaining and not grant dependent. (More about this in another post perhaps). I am also against pilots (not ones that fly planes, or medical trial pilots). Also with the mass of data available I think that we need to revamp statistics - sample sets are less important - more important is noise reduction, or rather discerning salient information (trends) from the data, rather than extrapolating trends from a sample. (Again for another post)

Anyway, in speaking with Arv I remembered the importance of ethnographic research in the implementation of any solution. Many times projects fail, not because the data collection methodology is incorrect, or because the statistical analysis is not sound, but because people collect data incorrectly.

I was reminded of the passage in "American Caesar" a brilliant book by William Manchester about Douglas MacArthur.
When visiting Japan (before WWI), the Japanese generals said that there was a problem with malaria among their men. The men had prophylaxis (pills), but were still developing malaria - obviously something was wrong with the pills. MacArthur laughed and said something like "If I gave my men pills with instructions to take them every 4 hours - they would be dumped into a ditch and forgotten." The Japanese generals were horrified - our soldiers would never do such a thing. The next week a new batch of pills arrived with a new label - the emperor wishes that you to take a pill every 4 hours. After that there was never a problem with malaria.
No need for any fancy solution involving cell phones or alerts or sharks with laser beams - just ethnography (Arv liked this story too)