Feb 242017

Out of the Shadows of a mine (Beaconsfield, Tasmania)

A former (mature age) student of mine had, for years before I ever taught her, written and kept a variety of poems.  The first batch were originally written in a mining town (pictured above) that once, in its glory days of digging out gold, had even hosted film festivals.  Now it offers tours of abandoned gold mines and real estate is cheap.  While these poems may have languished for decades, they have now been published along with others of more recent vintage.

The student is Jean Grosse. She’s now in her 70s and she published them on her own under the title of Sparrowlands One.

The publication, at Lulu.com, of both printed and eBook, reveals how gutsy determination and lively imagination can produce poetry that enthrals readers while also giving its author, a new lease on life. The poetry released from these old storage boxes remains fresh and poignant. Poetry can pierce the heart and yet it leaves no scar. In this book the poems heal emotional scars while also framing moments of ecstasy.

Diversity of topics

The book contains 32 poems, each with a photo or illustration and a brief biographical introduction. The poems span the years from 1965 to now. The poems cover a wide range of topics from the ravages of war, bushfires, and child abuse; the tender intensities of love, marriage, and music; the wily ways of possums and teachers; the fetish for credentials; the complexity of golf; and the impact of cars on toilet training and much more! The breadth of topics does not mean that the depths of the soul are not plummeted. These poems provoke compassion, sorrow, indignation, mirth, laughter and wonder.

The sheer variety of topics, forms of expression  and associated photos all highlight Jean’s tenacity to produce the whole product – writing, formatting, editing, layout, and final publication of both print and electronic versions. To do all this, she had to learn to work with Google Docs, eBook formatting, Lulu’s publishing system, and collaborative editing.

Poetry as Therapy

The sudden death of her husband, almost 4 years ago, prompted a reworking of poetry already written as well as the writing of new poems. She expresses this situation in a moving poem called Loss and Recovery – “I can no longer hide in the seductive / folds of flurried days of no consequence”.

Part of her mourning process also involved reading the recent and magnificent re-translation of Dante’s Divine Comedy (a poem of 14,000 lines) by Clive James. James translates the Italian poem into a pacy and even racy Australian idiom, which compellingly reveals the redemptive thrust of Dante’s original poem. Jean has been an avid reader of James. Sparrowlands One shows signs of being inspired by the poetic practice of James, while the whole journey of the Divine Comedy enabled Jean to review her own life against a vast canvas of emotional textures and settings.

Printed and eBooks versions are available at Lulu.com – by searching for ‘sparrowlands’ on that website.

The power of poetry illustrated

As poetry acts on emotions, merely talking about poetry in general risks deflating it into hollow commentary: the power of the poetry works by reading it – preferably aloud. Only a plunge into a poem can reveal how the poem rivets the reader’s attention and often releases lost memories. Here’s one short example from Sparrowlands One, and some specific comments about how each stanza works and how the whole poem hangs together. The poem recalls the return of a young 24 year old from the Great War, exactly a century ago. Later, Jean was one of his daughters who arrived later.

Fremantle, Before War’s End (1917)

The stretchers and wheelchairs emerged first.
Behind them, ramrod straight,
Sid forced his painful joints to march him
down the gangplank and home – a broken man.

The band struck up a tune – then stopped,
horrified at the slowly engorging procession of carnage. 
The silenced crowd watched,
many embracing their returning wounded soldiers.

Quietly, painfully coughing,
Sid marched with lowered head, 
ashamed of surviving a war still raging.

He knew that his were not the visible
injuries which had shocked this patriotic crowd.

He could not know that his marching would prompt 
a white feather in his mailbox on the morrow.

The geometry of return

The first stanza is about horizontal stretchers, circles of wheels, vertically rigid walking, and a downwards-sloping gangplank to Australia, at the far end of the world. Within the whirligig of lines, circles and movement, we get a name (Sid) and the pain (in his joints). These lines may also remind us of times when we may have had to ‘hold it together’ in some public event or procession.

Music of witness

In the second stanza, music captures a moment of the cascading flow of wounded bodies down a gangplank. The cadence of the words capture this flow. Then the sound surrounding the flows gives an outsider’s view: a band had  ‘struck up a tune’ bearing witness to and alas, was then interrupted by, the utter horror of  unrelenting flow of so many returning soldiers’ maimed and stretched. One can hear the last faltering puff into a trumpet and see mouths agape in shock. Here, the stanza can remind us of times when we avoid seeing horror by simply closing the blinds or crossing to the other side of the street.

Hiding pain

The third stanza of the poem now delves behind the name and the pain to reveal a hell: the shame of Sid having ‘got out’ sooner, than War’s end. The macro ‘war still raging’, in trenches far away, continues to reverberate through the micro ‘quiet coughing’ from soggy lungs. Are we alert to recognising when perhaps a twitch that tells of terror that could be momentarily calmed by a tender touch?

Missing the signs of sickness

The fourth stanza starts with an implied contrast between the visible injuries, particularly from war, that can produce such visceral anguish and outpourings of compassion with cases where the injuries are hidden. For Sid had no stigmata of external wounds that could frame patriotic actions on battlefronts far afield. Without witness, his hidden wounds further stiffen Sid’s joints that yearn to relax. On the one hand we feel compassion; on the other, the poem prompts us to review our disposition to probe any sense of plight that lacks high visibility. A simple “How are you?” can make a big difference.

Plight versus flight

The poem ends with a final punch to the being of a man already emaciated by war. This final blow – comes not from a crowd of close relatives or family at the gangplank – but from those mothers and sisters walking through the city streets of Australia, whose men had died or been badly wounded in the Great War. In these streets such women could be affronted by the sight of apparently able-bodied men returning before the war had been won, or worse, never having gone in the first place. Such men ran the risk of being given a white feather by a passing women as a public rebuke for cowardice: the men,  particularly if they had indeed gone to war, were left to recoil in anger, then reduced to bouts of defeated silence. Jean’s father, like so many other fathers then, carried all this pain in the solitude of shackled silence. Here, the stanza reminds us to hold back on quick judgments based on hearsay or convenient conjectures.

Why the poem works

Overall, the power of this poem comes not just from its theme (perhaps the poem may have been better titled as “Dying Man Walking”). The power also comes from the lack of predictable ‘rhyming’. Typically, we expect poems to rhyme in some neat pattern. However in this poem (and indeed others in the book), various rhythms and unpredictable rhymes crisscross the emerging meanings in the poem giving them more texture.  To reveal this, read the poem aloud and the heartbeat of the poem will elicit emotions and perhaps reorient the you to history, judgements, and the tyranny of distance and contagious disquiets.

But on reading this particular poem aloud – pausing at its turning points –  another theme also emerges from in the undertow of rhythms: a daughter’s homage to a father who had stayed so silent so long. In case you had not counted: the poem contains exactly 100 words - one word for each year since the event it captures happened. Such disciplines – from both father and daughter!

Buy the book!

As the eBook version of Sparrowlands costs just over $5 (US) on Lulu.com, this one poem reviewed here really is Jean’s ‘two bobs worth’ (i.e. 20cents)! Each poem in the book can be a tonic, far cheaper than therapy. Many of the poems offer a solemn yet heart-warming salute to an era of Australian history that does not need to fade so fast.

In these multi-tasking times when ‘staying in touch’ too often means getting phone calls anytime and anywhere, SMSs, Facebook alerts, emails, and social media notifications, poetry gets us in touch with ourselves and indeed others we could never have met. Good poems shift us from dealing with deadlines, dangling conversations and humdrum duties,  by recharging our capacity to find, and face, destinies.

As Auden put it, poetry can enter: “In the deserts of the heart” and thus “Let the healing fountain start”.

Sparrowlands One has heart and gives hope.


Feb 042012

Bon Voyage

Meet Barbara Potter – here dancing momentarily,  a few years ago at an Art Deco Cinema in Cremorne, Sydney.  Barbara died peacefully in hospital December 29 2011, after being admitted to Emergency only a week earlier. She was my mother-in-law. She struck me, particularly in the last week listening to her, as someone who really deserves a monument in stone:  I can but offer words that give some sense of the passion and love Barbara must have given to the students she taught in Tasmania. As with other posts I have written here, I have always sought to go ‘digging wonder’ – even within what might be dire situations.

Barbara had been a primary school teacher, working within the State Education system, across Tasmania. Let me start with a one revealing example that a former student of hers recalled: her teaching handwriting in primary school: Cord Cursive Writing.

The Dance of the Pen

You might recall learning how to write with a pen: tedious exercises, tracing outlines, writing between the ruled lines and trying to keep that pen flowing smoothly. Barbara’s method comprised: writing-to-music.  She would play, for instance,  ”Baby Elephant Walk” by Henri Mancini.  The movement of the music gave a rhythm to the writing.  And so, energetic children could be subdued by the beat of the melody and pens could then dance in unison across their ruled pages.  Check out, in the 2 minute video showing the rotating vinyl with this music.  Imagine yourself in the class.

Note the innovations: technology in the classroom, active engagement, fun, and real outcomes.  This was in the late 60s!  What gusto to try this in a class! Students would have had to ‘feel the love’ to give this a go. Some clearly remembered it.  Ironically, some of the same and earlier cohorts of such students did eventually come into University in Hobart where I had arrived (wide eyed from Cambridge University) to teach later.  Like her, I also experimented with teaching innovations, a similar one being teaching data-based writing (ie students writing about some graph they produced but with their computer screen off).

The Teacher Brigade

Barbara developed this pedagogical flair over many years, having started teaching at 19, in a little seaside town of Snug (south of Hobart) with four other female teachers sharing the ‘teachers house’ – all just out of Teachers College.  Even now,  some of these teachers wrote condolences recalling Barbara’s effervescence back then.  We tend to forget now how thousands of young teachers over the decades, went into classrooms, triumphant in the belief that education changed lives – by building up character and deepening the roots of competence, especially through the 3 Rs. And, in those days at least, they were correct.  (In the photo here, Barbara is at the bottom, centre: not much older than the students she taught).  My own mother was also of this ‘teacher brigade’:  having arrived from Italy with me in the 1950s, she also taught in primary school over the same period.  She too was resolute in her commitment to teaching, revealed by her perseverance despite not being made ‘permanent’ until she was 62 (due to School Inspectors persistently detecting ‘poor pronunciation’ of some English words).

Barbara would not privilege ’good form’ over substance. Whoever the student, whether from the farms, or the city and/or  in or out of poverty, (be it George TownKing IslandWinnaleah, Snug etc), they would get the same inclusive pedagogical embrace as anyone else. When, at one school the students were to be marshalled for the Queen’s visit, (remember those?), Barbara refused to take them. Teaching is not about watching a passing parade – let alone the display of monarchical vanities – No!  It’s about doing something for your destiny at your desk!

Teaching: Ongoing Vocation

After moving to Sydney in 2005 where her much loved daughter worked, Barbara became far more energetic – walking a lot, talking to many.  She particularly liked pampered seaside promenades (here shown at the Botanical Gardens with her personal parasol carrier, me).  She also bought new clothes and continued using her sewing machine to make more clothes.

She also rekindled her interest in teaching. Having long retired from the profession, she privately taught primary school mathematics to the boy next door.  She would go off into the Sydney CBD bookstores to find books with interesting approaches to teaching mathematics.  This interest in teaching mathematics was not some quirky personal hobby – she was part of a long tradition going all the way back to ancient Greece: You see the same passion in Plato’s Meno where Socrates draws a square in the sand and gets a little boy to work out how to draw another square that has twice the area of the initial square Socrates had drawn.  When Barbara met my oldest son - who had recently received his PhD from Princeton – she was quick to explore what problems he had teaching graduate student mathematics. It is worth mentioning that he and his younger brother (a software engineer) also came through the state education system of Tasmania.

Tasmania, a natural state of wonder?

While coming to Sydney had benefits, Barbara would still wax lyrical about Tasmania, although she would admit to the sociability of Sydney, its parks, and the comparative ease of  getting out and about.  Tasmania still held an enduring fascination.  Barbara wrote letters and cards, always in  exquisite  handwriting. On the occasion of the death of the father of a dear friend of hers, she wrote to him recalling aspects of the Jordan Valley (near Hobart).  Although she is describing someone else, her writing is as much about her and her sensibilities (then in her 60s):

To be able to hunt grasshoppers on the hill behind the house; to go fishing in the Jordan River in the days before the river was pumped; He [the father] even allowed his sons to keep ferrets!

While children played, a world of work whirred furiously away,  where even one man could do mighty things with gentle strength and a reliable truck:

He …grew Flax with its beautiful blue flowers.  Your father’s care of the green lorry that carted the flax to [town]: and he was very adept at the construction of circular hay stacks. [He also] drove the school bus … and rode the motor bike home. And how exciting it all was when the machine cutters visited to turn oats into chaff….
I was very impressed with his method for turning cream into butter.  And the fowl didn’t know in those days that they were “free range” and privileged chooks.

And while it might seem that such idyllic and ludic settings refer to an insular culture, it was also an accepting culture of mass migrations from far away places:

In the late 1940s there were a large number of immigrant workers on the South Eastern Water Scheme.  He made friends with them all: Latvians, Hungarians, Italians, Poles. Most could not speak English and he took time to supply their food needs. He was a “multi-culturist” well before the word was used extensively.

Such openness and inclusiveness came from cultivating the mind:

Both your parents valued education: books to read, encyclopaedias to browse through, old school magazines to read in the room upstairs…. Their role models extended beyond their own immediate family. I know I profited from sharing in their family life.

Even though these words come from a time before Barbara came to Sydney, they capture how she continued to talk about Tasmania. Some of its locales and locals remained dear to her heart. Having worked there in academia myself for nearly two decades, I too was in the thrall of Tasmania’s allure – even on one occasion writing an upbeat article about the wonders of Hobart and its people, in one of those in flight magazines that do features on  ’cities’.

Finding Peace

Over the last two years Barbara had become forgetful and on occasion disoriented (eg catching a wrong bus or thinking she might go ‘back to work to teach’), though she did usually manage to re-orient herself. In the evenings, occasionally, she could also get anxious – at times imagining that something of hers had been stolen.  It is ironic that when children falter over a forgetfulness or find themselves inexplicably anxious, we naturally help them to get their bearings. Barbara had excelled at this.  With older folk with the same behaviours, our care can become more circumscribed. For Barbara, it was becoming distressing that conversations that could dance and dart amidst ambiguities could now be left dangling – due to a lost word. Or worse, in their eagerness to help, people might ask questions – and questions were much harder for her to interpret than reacting to a clear statement.

After a happy and sociable day out on a sunny Tuesday (22 Dec, ’11), she had a particularly intense turn of imagining that her wallet was stolen.  This time, she could not be calmed and was admitted to Emergency which only agitated her further.  By the time I got there, she had regained her wits and wanted to leave. Unfortunately,  the intensity of the episode meant the ‘admission’ was well underway and not something that could be readily reversed.  The next day she was transferred to a secure ward for dementia.

Over two days, she realised her predicament – inevitable institutionalisation.  She reduced her eating, slept less, but  became very focused on talking about her life in retrospect,  as she had focused on students’ lives in prospect. Though her body strength faded fast, her spirit steadied to a calmer disposition. In the process she would talk or chatter and would tell me her age, increasing it each day (in her last conversation with me she said she was 99).   She was particularly impressed by the fact that the span of my hand could reach across her forehead and massage both her temples at the same time.  After such massages, a door would open to the past, out of which memories would emerge  - with the eagerness of children running out of class at lunch time.

By the week’s end, while on her own, she stopped breathing but was revived though remained unconscious. A priest was summoned on the same day to give her the Last Rites, by which time her husband, daughter and myself were there. As the priest blessed her forehead and said his last ‘Amen’ she responded, seemingly on cue, with a final breath. The End.  It left us all dumbstruck. Was this the hand of God looking to ‘give a sign’ through its timing?  Or, did it reveal the finesse of a woman who really knew that the rhythm of life finally demands – a well placed full stop?

Her leaving our world was so sudden, I have to wonder –  Did death claim her or did she claim it? Here was a woman unafraid to peer into the darkness of doubts, recall regrets, savour past joys, or share the thrill of witnessing her daughter thrive.  A twinkle in the eye communicated her resolve simply to let nature take its course.  She would let herself glide along that course,  allowing the buoyancy of her own words to take her to her final destination.

A final sign?

The funeral a week later gave Barbara a chance to conjure a final fling of fancy.  After the incense-filled Requiem Mass, the hearse moved slowly to her final resting place. As we followed in our GoGet  ’people mover’ (carrying her daughter, sisters and brother-in-law), I turned on the radio to find it playing:  ”It’s only a paper moon” – written by Harold Arlenin in the year she was born (1933) and rendered as a New Orleans jazz song played at funerals in that city she had so enjoyed visiting in 1981. (The specific version on the radio was by Ken Coyler’s All Star Jazz band).  The carefree song dispelled the pall of solemnity that hung over the passengers:  relief and joy burst into the car as the song’s tune reverberated through the trees outside.

But the words of the song were also compelling: about how enchantment only works if you  believe in the one who is trying to make the change.  How fitting! How is a teacher to teach effectively if she is not believed in?  How can an old woman complete a sentence graciously if we don’t believe that a valuable thought is actually trying to find expression? How is anyone , who is starting to feel like a burden, going to reveal their incredible lightness of Being, if we dwell on their maladies and not their majesty?  Listen to this rendition (with lyrics) from Ella Fitzgerald and remind yourself of Barbara – if you knew her.  If you did not know her,  let the song convey to you a passion for life and learning that so marked Barbara’s life even in her final poignant predicament.

At the moment when I realised that the song contained this line: “Without your love, it’s a Honky Tonk Parade“,  then I knew I had to write this post for Barbara.


Barbara Potter (nee Goldsmith) is survived by her husband Rennell and her daughter Karen. Barbara is buried at the “St John of God”  section of the Rockwood Cemetry, Sydney.

Apr 222011

It has become a truism to assert that we are witnessing an information explosion; that we suffer info-glut or information overload.  Relief apparently can be had from either filtering or summarising all this information or storing it in an accessible form for later use. I see the problem as one of a data deluge and an inability to allow ourselves to be informed by it.

The other night, I gave a presentation to the:  Institute for Information Management (IIM), Sydney Branch. My talk was part of a set of two speaker talks collectively billed as:   “Getting Up To Speed with Text and Data Analytics”

The talk gave me an opportunity to view information not as a ‘stuff’ to be summarised or filtered but an event to be unleashed.   In giving the talk and in subsequent and lively discussion, a number of insights emerged which I think I can share, and in so doing -  clarify.

Historical Parallel – the abacists vs the algorists

The image that came to me after the discussion was from an earlier time, in Renaissance Italy, when battles raged over two kinds of  ways of dealing with the recently developed financial data (double entry book-keeping being one of the great inventions of the times).  The protagonists of this battle were between two types of ‘reckoners’: the abacists, who used the abacus, and the algorists who used the  Arabic mathematics called  ‘algebra’.  Algebra  caused much consternation at the time – it was so different – even allowing ‘nothing’ (in the form of zero) to exist!   The woodcut above depicts one of these competitions with the ‘spirit of arithmetic’ overseeing matters.

At it turned out, the algorists won the day, even if initially their reckoning took more time. It turned out that abacus-based calculations of compound interest on loans produced under-estimates of correct interest amounts so the merchants took to the new mathematics, even if Universities at the time lagged behind!  I came across a cute anecdote of a German merchant pondering, back in the 1400s, on the future of his son and his fellow merchants recommending that if he wants his son only to be able to add and subtract, then German universities would do, but if he wanted his son to use multiplication and division, better he go to the merchants of Venice!

The deeper reason for the success of mathematics over method lies in the generality of the former and the false confidence ‘methods’ create: you follow the rules right, so the results must be right, no analysis, intuition, or wider views are required.

The cockpit of illusions

Now we have, again, a new kind of abacist, the ‘information professional’ who yearns to deliver information on-time, any-time, for whosoever with, of course,  the right platform – ‘abacus’. Information behaves like water – apparently to be filtered, pumped around, available ‘on tap’ and ready to quench the thirst for insight and oversight.

Current Business Intelligence promise ‘cockpit’ views and dashboard plasma screens of entire enterprises,  where managers can be made to feel ‘in-the-know’ as they survey the panorama of orchestrated business processes without the smell of sweat, treadmill of toils, or the sound of alienated voices, emerging from ‘below’.

Many an information professional has been lured into the view of being a handmaiden to such data-based chimera, made all the more alluring by virtue of the new breed of user-friendly data-analytic and visualisation tools.  Apparently  the user can  point and click their way to deep data derived discoveries! If only data were so ….. serviceable!

The ascent of records management

By good fortune, the people at this talk were not this ilk of ‘information professional’.   They displayed the kind of patience that comes from operating with gritty, even grotty,  information resources that need to be cleansed, marshaled and then accessibly housed.  The shift in thinking I wanted to explore lay in admitting to a data deluge, not an information overload with the ramification being we still find it inordinately difficult to release all the information contained in data as a proliferation of self-propagating events of ‘informing’.

Core concept: Information as event, not substance

Information, as a noun,  I think makes sense to see as describing a rapid change,  like ‘explosion’.  It refers to an event, not a stuff.  Just as explosions dramatically alter physical form, so information dramatically alters semantic form, that is, what we believe, perhaps without even thinking.

The verb form of information (to inform) captures this nicely: the difference between telling someone something and informing them of something lies precisely in the assumption that in the latter case, the person communicating takes the effort to check the other has indeed understood what has been communicated.  An informed person in thus ‘re-oriented’ to what is going on. The aim of the game might be said to be to “inform beliefs with believable information”.

Sometimes explosions just happen, just as sometimes,  situations surprise us. But the chemistry and physics of explosions has led to the creation of explosives:  these are ready-made to explode,  but only when detonated.  We need an equivalent term for information: I suggest informatives.  I use informatives to refer to carriers of information-as-informing-event. Just as an engineer might, on seeing some obstacle to an objective, deploy well placed explosives, so when we see obstacles to insights – typically in the form of confusions, misunderstandings, prejudices or opinions, we place ‘informatives’ to dissolve them, so that beliefs are better adapted to the situation. Part of the challenge here lies in showing that ‘informatives’ is not simply a fancy name for ‘fact’.

Much of what people mean about having, even too much,  information really refers to having informatives – just as the military might refer to having explosives, rather than explosions.  With some flurried pen movements over a whiteboard, I tried to depict, four distinct informatives:  Experience, Expertise, Data, and Fidelity.  The impact of the informatives ‘going off’ might be so called facts, but we need to focus on the capabilities, not their activation.  And just as explosives may pack a big or little punch or just fizzle out,  so too, these informatives may or may not inform. The informatives cane be ‘empty’:  people can get so focused on an intense experience they dwell on its intensity rather than what it is about; experts can get so full of themselves that they imagine they can pontificate on some matter without really delving in the the nuances of a situation that has posed the question in the first place;  data itself might just be noise; and finally fidelity may draw on transcendental sources making it seem answers can be deduced to any questions (eg fundamentalists).  Here’s what I was trying to draw on the whiteboard:

Data will inform to the extent it has been captured in records containing relevant fields that describe important objects within the flux and flow of events.  These days the vastness of data collections has yet to hit home.  Data are mute as they only contain nouns (objects) and adjective (fields). Putting verbs back in (ie verb-alising data) requires particular kinds of graphs and associated elaborations (para-graphs).  Experience builds on similarities between objects and events (rows of a data table). By contrast data informs expertise through differences and similarities found between fields (columns).  The more data can be modelled through analytical processes, the more it can inform expertise.  The issue of which fields matter and which objects are worth tracking and tagging comes from fidelity – abiding and collectively relevant concerns about eventualities.  Fidelity defines not simply a question, it motivates a questioning quest. It informs us of what we need to be informed about – ie our ignorance and our sense of commitment that we can get to the bottom of the issue with enough resolve, cunning and imagination. (Ironically, fidelity has too often been seen, wrongly I think, as a set of ‘answers’ – when that happens it cannot inform since it does not marshal ignorances into a common plight).

Illustration: Ambulance Data

My talk referred to a few more practical cases to illustrate the above, but in this post, I wanted to clarify (to myself at least) the wider issues for data analysis in and for organisations.

The speaker who followed me Paul Middleton, gave a superb example of the problem I sought to describe. I didn’t realise how large an operation the NSW ambulance has become.  The ambulance service collects massive amounts of data, mainly for billing purposes.  Every call out involves data collection. This same data could yield insights into many areas of health:  how different kinds of interventions (on the run) work in different settings – different pain killers, oxygen delivery, supports, ambient sound, monitoring equipment and communication with the hospital along with all of the contextual variables relating to patient demographics, locale, and injury. And Paul’s summary of analysis offered some marvel of good classical statistical designs – for instance trying two different approaches to a problem and comparing which was the more effective.

Some impressive results were shown which means more lives can be saved – for instance the use of some interventions at causualty can be applied by paramedics at the very site of the accident producing better outcomes. Hospital care can then be less intensive (expensive).  Insurance companies and governments save money.  Beyond that, more virtuous loops of support can then unfold:   Part of these savings could support more data acquisition and research. Para-medics could then not simply give out care but also take care to collect patient data.  Currently para-medics focus, naturally enough, on the patient, apparently even  jotting down data on their gloves which they transcribe later onto relevant forms once they arrive at the hospital. Some of the current value of the data in informing health practices could also fund data acquisition devices – from bar code readers, voice recorder, digital convergence of data from instruments to one storage device and so on.   All that data can illuminate far more than how much each case cost.  The data summarised in terms of co-variations can clearly inform medical expertise – that provide more nuanced causal mechanisms.  The data, displayed as a series of alterts, indicators and visualisations (of healthy functions) inform the experience of nurses and carers.  And these very para-medics, carers, in so far as they have an abiding fidelity to ‘health’ (which goes alas too often without saying) they can provide valuable insights into what else they would like to know via such indicators and their visualisation.

There are many software and data analytic issues to then operate in this way. I depicted some of them and have written about the software (R, application servers, messaging, and visualisation here) and data analysis here.

I end with thanks to all those that asked questions and special thanks to Brian Bailey for getting me to give the talk. I hope this post adds to what was discussed.

Apr 092011

Films cost money. Big money belongs to big players. What if a film could amass lots of little bits of money to finance its production?  The Tunnel, a thriller filmed in the underground tunnels of Sydney looks set to achieve this. You can buy a ‘frame’ for a dollar. I am the proud owner of 25 such frames (and can now claim to be a film investor!).  The film will be released in mid May and will be BitTorrent friendly – that is, rather than fight so-called ‘piracy’, the producers have sought to harness the love of film to finance it. Cool stuff!

The producers (Enzo Tedeschi - of My Small Italian Wedding, 2003 – and Julian Harvey ), Director (Carlo Ledesma) and a young cast of actors have re-opened the pathway to ‘patronage’ (from those days when rich, established families could provide patronage to artists – like Mozart or Michelangelo).  The difference here lies in socialising the patronage – small dollar amounts from many, rather than a large amount from a few.

Some rudimentary calculations on fiscal returns based on this ‘crowd-sourced’ funding model get outlined at Laurence Timms‘s blog.  The bigger issue lies in this film breaking some ground. If the film proves watchable, perhaps even gripping, others will try this model out – of mobilizing a large base of support around some concept that appeals and some production in which people feel they can trust.

The other interesting feature of the funding model lies in its ‘lottery form’.  Buyers of movie frames go into a lottery where a small number will then be entitled to 1% of the revenue.  Of course besides the lucky winner of that (probably modest) income stream, frames owned, I am assuming get ‘framed’ so that I will be able to display the frames owned (a small array of 25 frames).  In addition merchandise for the film can be found on The Tunnel website.

An associated challenge lies with deploying this model with other kinds of creations – sculptures, books, paintings, performance.  The last of these implies this, as we go to a performance and ‘pay’ for it there and then.  These new models move the payment ahead of the creation, shifting cultural consumption from what are called ‘experience goods’ (buy to have an ‘experience’) to so-called ‘credence goods‘ (buy in the belief that what you will get will work – eg going to a doctor or a mechanic or donating to a charity are all ‘credence good’ transactions.

Could it be that a film made in the tunnels under the Sydney CBD comes to bury the so-called ‘block-buster’ movie?

Mar 082011

Meet my grand-daughter, Jasmine at her first birthday.  With my birthday in the same month, we had a grand dinner – her dad paying for the lot:  So many accomplishments cascading through the generations!

Her eyes already show the mix of Chinese and Anglo-Italian makeup.  Her father, Paul, won over the in-laws’ hearts in China by giving a speech in Mandarin at the wedding.

Children disarm and yet impact on virtually every nook of life that parents had imagined they could preserve. The sociologist Peter Berger somewhere wrote about children as a perpetual invasion force that progressively subdue the natives (their parents) into taking on duties and altering their ways to meet the pressing needs of the invaders.  Even here, Jasmine trying to take little footsteps had many hands being offered to assist.

Now (6 months later) she can walk and run and as she discovers the laws of gravity with each fall, others quickly reassure her that falling is really mother nature’s way of  playing with her.  Wonder rather than fear seems to keep Jasmine chasing new horizons – both in the external world but more amazing, given her age,  the internal world of ‘rules’.  Now with the parents duly ‘baby-proofing’ the furniture – with special latches on doors etc, Jasmine herself has taken to inspecting each such latch and demanding any unlocked latch gets put right.

Her mother Lili seems to have developed that wondrous capacity to let children ‘find their feet’ rather than fretting over possible mishaps. But an interesting feature of Jasmine’s life lies in Lili having her mother,  Sui-Ping, living with the family, thus ‘in-sourcing’ child-care – at the very least.   And so this blend of Chinese inter-generation support combined with an Italian capacity to find fun even in the humdrum, means Jasmine’s get the love of her parents without them collapsing in exhaustion while trying to manage two careers and raise a daughter. Triumphs abound.

Feb 252011

This is a night picture of Seoul viewed from the heights of the Top Cloud restaurant.  Virtually right below Top Cloud, a “little” Venice (I would say “piccolissma”!) rivulet flows past.  Seoul appears to have  a massive number of dense and socially alive centres, where people enjoy company and of course drinking.  But what really impressed me was to drink without a worry about how to get home. Hence the title of this post.

Sometimes people come up with ideas so cool you wonder why you had not thought about them yourself. How do you solve the problem of going out in the city for a night of merriment and then trying to get home?  Normally we imagine either using a taxi/public transport or have one of the persons in the party not drink so they can drive.  We have become accustomed to add  a dash of sobriety to our thinking about drinking, lest we run foul of a breath test (‘Sobriety Police’).

How many times though do we find our preparation or assumptions about getting home go wrong? Public transport may not exit or if it does, crowds crush within. Or taxis provide speed and privacy, but can you find them at those times at night when you need them? This becomes particularly pressing when, despite the delights of the night, rain or cold weather assails you. Or the person who was supposed to stay sober, to ‘drive’,  gets carried away and has drunk too much. Not to mention the searching for the car with a hangover the next day!   Here’s a story of a different solution. A (non-governmental) public utility service without much of an infrastructure!

I was at a dinner party in the busy city of Seoul last week at a quiet place in the upmarket part of town.  Everyone was drinking at this gathering – even though they arrived by car (eg SUVs – Hyundai, of course!). So how, I wondered, will they get home? And more pertinently for me, how will they get me to my hotel?  Eventually after one of them made a phone call – which I thought was for a taxi –  a guy, a complete stranger to all, turns up and within a minute he drives the car owned by one in our group.  The car darted and drove through the traffic with the finesse of a dancer.  Passengers merrily enjoying the ride (in their own car!).  These guys know how to drive – fast.  In our case, onwards to yet another destination (my hotel would come next).  Then after that, another guy turns up and drives onwards to homes and hotels.

How does this system work?  It turns out there are thousands of these ‘drivers’ and driver companies. Email in-boxes in Korean offices are bombarded with spam regarding this service.  In my stupor I thought of them as Serenity Soldiers! People on call to rescue all car drivers from the perils of drink driving. I couldn’t work out how insurance works (I am assuming these companies have cover for accidents, or maybe policies have a ‘class of driver’ that covers such services?).

Why does such system exist?  Less public transport, more car use (again ubiquitously Hyundai), and taxi expenses – if you go hopping between places. So a private system has evolved which provides a transport safety net that operates through a market.  So the entirety of people at a table in a restaurant can go their merry way. A veritable vortex of allegria happens – everywhere. It appealed to my Italian sensibilities. (Posts to come on the Italian sojourn that preceded this part of a long trip).

Compare this to the tyranny of sobriety checks. In order to stop the small portion of  ‘drink drivers’ via breath test, entire cities have their residents either to transform drinking into an all out ‘binge’ event for the young, or the drinks are consumed with a cultivated abstemiousness (“I’ll have water now”) for the less young, or residents stay at home – particularly if older.  But if one person has to stay sober in order to drive, the very disparity of inebriation levels within a group alters the social chemistry. The risks mount of some people ‘making a fool of themselves’ and others to be able to – indeed feasting on being able to – remember this. It need not be like this.

Does Sydney have such ‘serenity soldiers’?  If only……

The Sounds of the Kitchen

 Comments Off
Feb 172011

Last night I went to see NANTA a non-verbal performance  in a kitchen setting.  The two hours will alter the way you witness kitchens ever again!

I have been travelling, (6 flights and 4 train rides so far) but only now, in Seoul with free broadband, can I get to write down what I have witnessed. (Other posts of earlier events & meetings to come!).

Last night my amiable guide Annie took me to see this performance in downtown Seoul.  To get to the theatre,  she guided me  through what looked like a department store, but at the back, sure enough, a ticket office pointed to the entrance to the theatre.  Seated in the very first row, seemed like prize position – until the show got under way: “up close and personal” does not quite capture the visceral sense of being in a kitchen.  The photo here I took from the my seat just before the show started: it shows traditional tables for preparing food.

Here’s a short review by (unamed) Singaporean

Cookin or Nanta is a delightful, comedic musical Korean version of “Stomp” that transcends the language barrier. It combines Korean folk music and the pulsating rhythm of Samulnori, or traditional Korean drumming, in a crazy, fun way. Nanta debuted in 1997 and has since performed internationally in more than 141 cities. …. The performers had everyone in stitches, but audience beware: particularly those seated in the front, you might get more than just kimchi on your shirt front by the end of the show.

Percussion paces the show through the sounds of knives hitting a chopping blocks;  sieves rubbed with whisks; rolling pins hitting tables; pots and pans pounded or gently tapped; and of course the sound of food being cut, cooked, packed, and presented.  The show will produce for me new associations whenever similar sounds in the kitchen occur any time soon in the future

For the show to run for 2 hours would seem a stretch. It might then be described as a long drum roll (with proxy drums).  But the very fact that it does run for two hours,  points to something else that keeps it alive. While  percussion provides the pacing, plot and characters give a lot more depth to the show.  Three cooks find they have to prepare a wedding banquet in an hour, by a manager who, apart from giving this deadline,  installs his cousin as a chef (even though of dubious culinary competence).  So the five characters in interact in a flow of one-up-manship, slap stick and energetic dance routines amidst the tables and cooking (which at times has flames, water, steam all impacting on all real food being actually cooked). All dialogue mimics language rather than being real dialogue. Names of food (in English) find expression in song. One particularly entertaining routine in which the cooks sing these four words -  onion, cucumber, carrot, and cabbage – initially in a contrapuntal arrangement  then in a 1950s style rock arrangement ought to be added to the collection already on YouTube!

Being in the front row felt like those 3D films where stuff comes at you.

First,  dance that close up becomes more of a vortex pulling you into a swirl of bodies and muscles.  With a stage that wide, I could not see all at once, so literally a performer would leap past seeming coming out of nowhere.

Second, the props – streamers, brooms, knives, pots, flame throwing etc – actually crossed the edge of the stage at times: You could feel heat from fires, sense the moisture of steam, hear the huff and puff of dancers breathing and the sound of food being cooked.  And the drumming, particularly at the end when massive drums, camouflaged as barrels get ferociously pounded, literally had chairs, bodies and floor vibrating. Some drums had water on the top of them so all this water would sparkle up on each thump made all the more compelling with the strobe lighting that would occasionally intensify the routines even further.

Finally, the play moved towards more audience participation, with audience members invited onto the stage to participate. Thus the boundary of ‘performer vs spectator’ become ever more diced, just like the food on the stage.  The show ended in wild applause as much for performers as for the audience themselves. In the end the sounds of the kitchen had left their mark.

Jan 152011

If you go watch The King’s Speech, it will most likely move you in ways that, on reflection, will prompt the question “how did that happen?”.  How can a story – of a guy’s stuttering and his speech coach trying to help – become that gripping?  The advertising of the film and its reviews tell all that the film renders a time with effortless precision and that a story will unfold of the unmaking of one King of England and the making of another, utterly reluctant King.  Even so, the emotional impact far exceeds what such period pieces deliver.  Here I explore how this film delves into the depths of what it means to ‘have voice’, an issue that resonates today.  This film got to me:  and I noticed not just me.

The Film’s 4 MPs (Main Players)

To get a sense of the structure, and indeed cunning of this movie, look again at this revealing picture from the movie.  What do you see?: the Microphone (so new and daunting then!),  the Monarch (King George VI),  a Man (a coach),  and  the first of several Manuscripts (in this scene, Shakespeare’s Hamlet).  The film swirls around these four points, each move, opening up another aspect of ‘Voice’ – that flows and  gets, by turn:  frozen, fragmented, fouled, fooled, frenzied or fetishized.   But finally,  Voice flows forth with a force-of-feeling strong enough to frame the forebodings (in 1939) of an entire Empire. Listening to the original 1939 speech, available at the BBC archive of the actual King, all of a sudden each pause packs a punch: what would otherwise sound like pretentious pauses and a lisp,  now sound like a real struggle with the monarch trying to rally all to a much bigger struggle.

While the vast majority of the reviews rave about the film’s many qualities, few dwell on the intensity of reactions.  Here’s one revealing example from the Financial Times (5/1/2011):

The Lures of The King’s Speech

You can settle into a viewing of this film from a number of angles: as a ‘period piece’ with sumptuous clothes and decor that take you back to a finer time:  “Ah what a time that was…“; or a family drama – between brothers which connect with family types you may remember: “The talkative cad who chatters endlessly saying nothing,  versus the brooding stiff with so much to say yet no clears words express…..” or the family dynamics between father and sons may equally remind you of family dysfunction: “That father, wise to the world, yet so frozen to his family…” ; or  maybe the film can uplift you as a ‘true story’ of heroism – of difficulties and disabilities courageously confronted: “Wow, such ferocious determination and final overcoming…..“; or as war film, whereby remnant nationalism (akin to Last Night at the Proms) bestir you:  within the microcosm of his Majesty’s household, cloistered mannerisms give way to the marshaling of meanings broadcast across the British Empire to mobilize all – to confront the next Great Conflagration: “From pomp to pucker!”

Films with multiple angles and multiple meanings risk becoming a pastiche of plots or a labyrinth of hidden messages or a collage of cliches.  The King’s Speech creates and then controls multiple facades through a deeper, more compelling, logic.

Harvesting the audience’s imagination

How can this film be a ‘period piece’ supposedly giving us a ‘feel for the times’ when most of the film takes place in a basement apartment with peeling wallpaper?  The scenes of sumptuousness radiate so alluringly precisely because they provide respite, and contrast to,  that basement apartment and the ordeal of re-learning it contains.  Similarly, the family drama weaves its way through the film, yet the film gives just enough cues to elicit projections of what we suspect (about Edward with his fumbling with things – phones, cigarette, champagne bottles and glasses); the father’s sternness alluded to by his outbursts about “Come on man, just get it out!” when the other son (Prince Albert, ‘Bertie’) stammers and stutters and stumbles on his consonants and sinks in his vowels.  But the film has already got us rooting for the Bertie because we figure: surely we can partly blame the father.  The sheer anguish of listening to a pause with no end, makes our stomach churn and we almost want to cheer when some improvement happens (lest we be put through the pain of yet another pause!).  But the King never completely overcomes the stuttering – he just learns to manage it.  The whole war angle – with Churchill’s appearances and film clip of Hitler – makes us connect what we see to the many narratives we know so well: of the forthcoming WW2, the Battle of Britain and the like.  While war looms, few big signs of its arrival appear in this film (one siren plus one scene of sandbags).

Indeed the first secret of this film lies in how it actives and then harvests our own memories and projections that fill out details. The film, because of it absence of ‘special effects’,  densely detailed scenes leaves much to the imagination.

The film has outsourced its ‘effects’ to the viewers’ imaginations. Ironically, the film is primarily about absences, eagerly filled by audiences keen to witness and yet wipe away dark secrets of their own post-Empire condition. Unleashing imaginations require strong structures to pin down the resulting associations. The above picture shows the four key pins that peg down the ballooning images this film conjures. The are the four key actors of the film.

The Microphone

The first piece that holds all this together appears in breath-taking wonder in the very first scene. Reviewers mention the first scene of Prince Albert Bertie, at a race course, trying pathetically to give a public talk  that flounders in painfully witnessed stutters and silences.  But prior to this scene, a far more significant scene sets the stage so that the full measure of the disaster can be understood: a BBC announcer, back at BBC headquarters and who will introduce the Speech,  walks into a special room within which, on a desk,  a massive bulbous structure (like a horizontally placed vase) sits in splendid isolation, namely The Microphone. It dwarfs the announcer!  The announcer does all these preparations with relish and fanfare, to ‘ready himself’ to speak  and then, with the finest of English diction, goes forth with his announcement: “This is the BBC, Good Evening ….. “  you can feel his words penetrate this microphone,  creating ricocheting vibrations within the hidden space of the microphone, eventually to bounce forth, as radio signals, across the Empire.  The Empire finds its most iconic expression in neat arrays of radio transmitters each with a country’s name on it.  For this announcer,  the microphone offers a portal to the world.  For Prince Albert ready to make his special speech alas, the microphone appears as a bottomless pit beckoning for words he just cannot yet muster! The film pivots on the microphone.

The Monarch

With amazing acting finesse, Colin Firth, playing “Bertie” (Prince Albert), palpably captures the sense of compressed and compacted meanings – busting to be released, but he just cannot do it. He is not yet king. He presumes he will not ever be king – his flash and brash brother will become king.  Even now, their farther, the King, points out in a telling aside, that the monarchy has been reduced, thanks to The Microphone,  to the most demeaning of profession: acting! Monarchs have to ‘connect’ with the feelings of The Many. The disdain and resignation of the King makes it all the more futile: Prince Albert must speak into the microphone, till he dies – just for duty (forget delight!).    Yet from this high office and particularly with the looming clouds of  war, only the Truth ought to be spoken from such high office.  Prince Albert knows this and chokes.

A Man

Prince Albert has a wife.  The Microphone demands ease, quick wit and flow. Prince Albert has to get help.  A hilarious scene occurs where an old-guard speech therapist has Prince Albert try to talk with marbles in his mouth, a remedy recommended in ancient Greek texts. Albert’s wife ponders on whether it even worked then. She seeks out more practical solutions. She finds a therapist (Lionel), played by Geoffrey Rush: an Australian in London getting school kids to overcome stuttering.  Lionel plays mascot for the common man, nicely conveyed by his Australian, no nonsense, laconic ways.  Impressed, Prince Albert’s wife convinces her husband give it a go with this Australian therapist.  Funny scenes unfold as English reserve, restraint, status and inner fury meets witty, unrestrained, role based flair at getting Prince Albert to engage – as a human, not a stuffy incumbent of some high office.  Amazingly, it works: for a rare moment (captured in the above picture), Lionel uses Mozart (played loudly in the headphones) to distract Albert’s searching (and searing) self-monitoring and he can read flawlessly from the manuscrpt: “To Be or Not to Be, that is the question” (from Hamlet). Nice touch!


Where-ever microphone appears, so too does a manuscript.  A sub-theme to the film (which primarily concerns Voice) relates to ‘reading’.   A ‘talk’ does not just get spoken, it gets written first, rehearsed, rephrased, etc and then finally read.  But the epic struggle in this film actually lies less in the words spoken than in the pauses that have to punctuate the delivery.  Each pause can collapse into an abyss of silence.  So Lionel and the King get the knack of filling the pauses with lashings of all ‘anti-scripted’ speaking: swear words,  song, arm waving, even dance!  For normal speakers, reading a sentence just happens: the Full Stop, at the end of a sentence, but a tiny reminder to take a breath.  But for Albert and anyone who feels the power of words, that full-stop looms ever larger: like the plug in the bath that once threatened to engulf the baby in it.  The film reveals a glimpse of the final speech notes and they are full of annotations – a support to counter the punctures that punctuation can deliver!  But the final accomplishment for Lionel lies in him becoming a human embodiment of punctuation: He silently mouths the punctuation.

Finding Voice

Just as the film Amadeus managed to make Mozart’s music almost a character in the movie, so too in this movie, voice becomes a character.  Voice here means more than sounds. It means a smooth flow of sounds that signify something significant.  Not only does Prince Albert find a voice, the whole Empire finds a voice.  A voice that resonates with the both the pain of having to say what has to be said and also saying it, such a voice that ‘moves’.

Today, the sound-bite, gives us an endless flow of little nuggets and asides that fills a gap, like a piece of candy gives comfort. Yet no nourishment happens.  Voice takes up time and makes time to engage.   Firth does an excellent job showing this. And Rush reveals himself as the mid-wife of Voice: he shows a solitary brooding can never make for a speech (even if Shakespeare indulges this myth at times).  In the end, the King’s Speech is less about a King and more about the many times we find people waiting for someone to say something about an issue and we balk and barricade ourselves in the quiet but false knowledge that someone else can say what has to be said. The courage talked about both in the film and about the film, covers as much the confrontation with a disability as it does a confrontation with willfully not speaking out.

Even more relevant, in an internet age, where a good idea might replicate and bounce across a lot of places, the value of a genuine voice becomes all the more compelling.  In this age,  the ‘microphone’ or indeed the camera no longer inhibits or  causes the fear, instead the dispersed swarms of search engine ‘bots’ that capture all and store it forever haunts writers.

Jan 102011


Last Thursday, I gave a talk – at Google Offices in Sydney for a monthly meeting of the Sydney Python Users Group SYPY – on an emerging architecture for aligning a range of components that would enable statistically powered web-sites to operate easily.  I gave an earlier version of the talk at SURF (Sydney Users R Forum) in December.   Unlike having a database powered web-site or a real-time web-site mashing up web services, a data-powered web site needs to ‘crunch’ and combine a large amount of internal and external data to deliver results to users.  The crunching typically takes time.  Statistical algorithms can hog lots of computing resources. Their results may even need to be rendered to the reader as interactive graphs, so a reader can ‘model’ different options, seeing the graphs change accordingly.

I introduced the example of trying to have a web site that could report your contribution to traffic congestion. With moves to have registration fees for cars linked to how much they are used during congestion, some such web site will emerge.  For this to occur, the web site not only has to show how much driving you have done but also how many others were on the same roads at the same time.

Another example relates to carbon footprint: suppose a web site shows you not only how much CO2 you have been producing but whether you outperform or under-perform some benchmark you have selected (for instance some peer group).  Similarly, exercise groups might compete with each other or individuals within groups. In most of these cases, the results do not need to be instantaneous but they can become quite complex, particularly when you begin to ‘weight’ for, or adjust performance, by various characteristics.  For instance, if your car is particularly small, then its contribution to congestion might be adjusted down; or as an older person, your projected weight loss trajectory is adjusted to be made comparable to younger trajectories.  Note, in all such cases, some kind of oversight group will typically exist that needs summary overviews of the whole situation (traffic, fitness, weight, or whatever).

To serve both the user and any oversight entity,  a statistical ‘engine’ must come into play.  The R programming language provides a powerful framework for transforming data into information a web site could deliver: data may need to be cleaned, transformed, analytically dissected, missing data interpolated, predictions extrapolated and error bands (of uncertainty) defined.

The talk produced a lot of discussion and, for the first time for me, someone blogged the talk as I was giving it!  See Net Traveller.  Thanks  to all who turned up for useful questions and a prompt to write this post-in-review.


The title of the talk, gives a clue to the architecture and the actual deployments possible:  Making R RESTfully enterprising with Django. Because R focuses on statistical operations, and the Python language has such a rich library of mathematical packages, an ‘elective affinity’ exists between these two languages.  In this architecture, Django , an application server written in Python – pitched as a framework for perfectionists with deadlines -  ‘exposes’ R to the web (or an intranet) as a ‘service’.  This means other software programs and not simply individuals, can invoke R to do some statistical magic, consuming the R output as required.  The other side of this, of course, means protecting R from an onslaught of requests. R gobbles up a lot of memory and processing. Queuing the requests provides one pathway (see below).   Django needs to ‘talk’ to R and other components and of course needs its own database.  The following criteria determined the selection of these components: Ease-of-use, vibrant developer community, open source, robust software and ‘plug & play’ ‘framework structure, low maintenance and easy access to ‘script-savvy’ personnel:

Basic Components

I’ll describe the basic configuration by starting at the bottom and working up – as subterranean spring might be followed to where it finally gushes forth :)

Vast amounts of data already exist on the internet.  We are witnessing a tsunami of data.  See Freebase for a specific instance and Open Data for the ‘movement’ involved and Gapminder for fantastic videos and discussion on data. Google of course continues to make data available through GData (eg Book search results).

Django can connect and ingest such data. It keeps such data in its own Mysql database (maybe after cleaning – perhaps via Google new Refine 2.0). MySQL-Python links Django to Mysql nicely and transparently.  Django itself searates out the ‘model’ (or objects it deals with, typically mapped from a database), a controller (which relates to its whole framework ) and views (which define what data are to be presented). Note: this so-called MVC design pattern in Django distinguishes between a View, data delivered, and a Template, data presented which, in this diagram, gets addressed ‘higher up’ the chain.  In effect, if there are work-flows to be followed in processing data, the Django framework can control all this. Often, many human interventions nudge data processing along: particularly when complex problems need urgently to be addressed (as I routinely deal with).

At various points, if data requires multivariate analyses, Django invokes R, via a nice mapping system called Pyper that opens ‘pipes’ between the python objects in Django and corresponding objects in R. This makes for elegance and clarity. Within a serious application of R, the services it provides would be delivered a R packages.  In this diagram, note the package itself has three ‘environments‘ (or nested contexts that define objects that ‘live’ there).  Functions transform data, however, with environments, these functions can be made generic and then ‘morph’ to suit the specific environment within which they are invoked.

With data, appropriately transformed, usually weighted in some way, Django can pass the data onwards to its controllers (templates).  These pass the data to the browser as a json feed, making the data available to the javascript language (managed by jQuery) that controls the browser.  While javascript as such can be a horror to deal with – particularly to make it work across different browsers, jQuery makes for great ease and compatibility, enabling users to see dynamic screens with sortable tables, drag & drop facilities, dynamic menus, and of course, content.  The remaining piece, the icing on the interface cake, lies with graphs.  Here Protovis (now superseded by D3 – Data Driven Documents) does an amazing job producing highly crafted graphs that, with jQuery, can become interactive, giving users a kind of customized Business Intelligence function.

Managing Requests: Messaging Systems and the REST protocol

The description above does not cover the interaction between other systems and this R service. It happens that other systems may have their own data and wish to invoke the R service (which in turn may grab additional data from other sources).  If you have the resources of Google, all could be made available as a service. But even Google draws the line at computing resources. Google cleverly designs its services to involve mostly retrieving data, not processing data.  Typically the above user cases, do not require real-time responsiveness. Rather they  involve some kind of  ’accounting period’ – within which each activity or user could have their own specified date for ‘re-calibration’.

Thus a first line of adaptive defense against swamping R involves using Messaging systems. These work particularly well within organisations (ie intranets, rather than the internet).  So if a Roads Authority did issue registrations based on traffic congestion and drivers could look up and model their ‘congestion surcharge’ and off-peak discounts, the system would need to queue (and indeed cue) the requests on R to suit both their priority within a wider schedule of routine runs.  Messaging and scheduling systems really ‘queue cues’ that trigger processes. The AMQP (Advanced Message Queuing Protocol) cleverly delivers  queued cues of processes within an organisation. An good overview can be found here. The implementation suggested here, RabbitMQ which is written in Erlang, not Python,  has an interface to Django – called Celery.  It works well. The more complex the workflow and the need to meld distinct results from different processes, the more AMQP systems matter. But they presupposes a central authority or organisation, rather than the wilds of the internet.

The internet itself functions of HTTP (Hyper Text Transport Protocol).  Despite functioning well, it took several years before reflective minds discerned, within it,  a deeper architecture, ie  REST (Representational State Transfer).  The entire internet, with all its servers, proxies, caches, look-up directories (eg DNS) can be seen as a protocol that moves information about resources around. Now this thinking has affected communication between computers so that, for instance, in seeking a result from Google, instead of going to the browser a piece of software can access the same functionality ‘RESTfully’ by sending the request akin to a browser URL except there are a series of additional rules about response (so that machines can re-direct themselves intelligently to other resources if their first request fails).  Here Django – Piston exposes Django as a RESTful service (and through it, R).

Rounding Out the Service Bundle – RESTful Services

What I have described above I have managed to make work at least in test settings. If an organisation were to go-for-broke on this logic, several other support packages would be relevant, though I have yet to try these.  These are on the right side of the above diagram.

First, any system that involves points, monetization, pricing, or harvesting behavioural profiles can be ‘gamed’.  This has less chance of occurring if organisations are layered, sluggish, bureaucratic, differentiated etc. The obstacles to coordinating ‘gaming’ or fraud become too hard. With internet exposed activity, particularly where  calculations based on streams of data, game become possible. In some cases, one finds recommendations in favour of it – eg SEO: Search Engine Optimization.  But in more mundane and focused settings, new kinds of fraud and anomaly detection becomes appropriate and a nice example of developments of frameworks for this is Picalo.  Much of this work could be done in R, but the author of this Python package has used considerable experience with accounting processes to develop a plug in framework for data analytic audits.  I have yet to get this going, but it looks interesting.

Second systems of the kind that model aggregations of behaviours will need to simulate future scenarios.  This involves more than simple predictions (or even complex statistical predictions).  Organisations often need future scenarios in which diverse options are contrasted using available knowledge. Even Excel can do simple version of this. Another Python package, SimPy provides what has become a mature package for simulation.  Thus, to use the traffic congestion example, if alternate usage rules, infrastructure builds (or widened roads etc), and pricing mechanisms were to be combined in an optimum way (to maximize flow and minimize congestion), then this package has the necessary analytical and programmatic power.

Third, individual participants to whatever processes delivered by the above system, may need tracking and their nature and extend of engagement monitored. Here one finds so called Customer Relationship Management (CRM) software.  At one level CRMs try to track how sales work. Django could do this directly. However when the individual has a relationship in which they aim to be transformed (for example, lose weight, become green, become creative, develop a family, start up a business) and this very transformation requires support, then CRM becomes critical and SugarCRM (community edition) provides an interesting open source framework for this kind of work. Not shown here, but of even greater long term significance, are VRM (Vendor Relationship Management systems) – these turn the CRM on its head: thus a web site that paced and paired training (calibrating and showing progress against various benchmarks) would naturally be of interest to the vendors of training materials, guides, etc. Rather than ‘advertise’ on the site, the very site could enable users to make requests (put options in effect) and vendor systems would aim to deliver – in competition with each other – what had been requested.

Finally, in so far as web content has to be managed, ranging from blogs, activity organisations, disucssion, papers, forms and document management etc, then WordPress combined with BuddyPress (to super-charge with social media) provide powerful frameworks in their own right).

The above four packages can be accessed via their own interfaces (API) or RESTfully or both. As yet these have not been bundled as suggested here. But I am keeping a watch on them in terms of the above architecture.

Enough Self-Clarification for now……


Jan 102011

At a talk I gave the other night which included a reference to the hidden benefits of using a Car Share system, someone came up to tell me that they indeed use a Car Share system.  The cost to them for using the car runs about $150 a month (about the same for me, though it varies a lot).  That amount, they obtain by renting out their empty garage, vacant by virtue of no longer needing a car!  So in effect by renting their garage they pay for their travel!

Car Share systems tend to get deployed near public transport nodes.  If people live near a train station, bus stop or ferry terminal, they can commute to work during the week.  However for extra trips in the evening or week-ends or holidays, public transport fails to meet such needs as easily.  Hence the value of Car Share systems.  By ‘monetizing’ the garages at such locations, that extra need – of discretionary travel in evenings and weekends – can be ‘free’ if income from garage rental equals or exceeds the monthly cost of such car travel via a Car Share system.