The web-site is now in readonly mode. Login and registration are disabled. (28 June 2019)

jonathan Vernon : the contents of my mind

Cloud created by:

Jonathan Vernon
13 January 2013

What tools, techniques and behaviours can be adopted in order to enhance and complement our memories? There are those attempting to record all that they do and say every day. Is this not the equivalent of hosrding? Is it a form of disorder? Could we record and editas we go along - or is this how our deliberateky forgetful minds work anyway?

Extra content

I'm looking closely at the academic work done on Virtual Companions. Also at research at Mircrosoft and IBM on data capture, storage and referencing - of the 'recording your entire life' then thinking what tools, software capacities and other activites can help edit this content to give it value and worth. 

Jonathan Vernon
22:50 on 17 January 2013

Automatically assisting human memory:

SenseCam browser (Microsoft). A wearable device that takes a picture every 22 seconds. Hodges et al (2006)

  • Tools for lifelogging
  • Hundreds of thousands of images grabbed and presented to aid memory ... and memory rehabilitation.
  • Automatic content analysis techniques

(There is a reason why we foget. The quote from James on the need to spend as much time recalling the record if everything is remembered is like that of Lewis Carroll and a map the size of the real world - neither had the advantage of limitless digital storage capacity and the ability to zoom in and out or back and forth - to expand time, not simply record it.). 

  • A visual record of your day. Berry et al (2007)
  • 2000 to 5000 images a day

Best practice

  • Only activate the device for significant events

Methods of review

  • Clustered time view
  • Geographical map (required GPS)
  • Interactive story authoring
  • Motion sensors identify events - typically 20-30 in a day.

PROBLEMS

  • Cognitive overload
  • Keyframe image selection a human endeavour
  • An entusiastic lifelogger might expect to gather 100,000 images a month.

OF NOTE

  • Key frame selection only of note if it picked a poor image. 

REFERENCE

Berry, E., Kapur, N., Williams, L., Hodges, S., Watson, P., Smyth, G., et al. (2007). The use of a wearable camera, SenseCam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis. Neuropsychological Rehabilitation, 17(4), 582601.

Hodges, S., Williams, L., Berry, E., Izadi, S., Srinivasan, J., Butler, A. et al. (2006). SenseCam: A retrospective memory aid. In UbiComp: 8th International Conference on Ubiquitous Computing, Vol. 4602 LNCS (pp. 177193). New York, NY: Springer.

Jonathan Vernon
09:59 on 23 January 2013

Research appears to be on improving the lot of those with dementia or recovering from a stroke. Whilst testing a product or service in such an environment will reveal potentials and value my interest is in extending human capability, therefore using the device to support and enhanced the capacity to see, record, store, recall, reflect upon and use accurate, objective data about events and circumstances. Over time, for example, watching an artist at work, or gathering the output from a lecturer from lectures, talking with colleagues and in tutorials with students would build a picture of thise person as an educator and may help model moderators with some AI functionality.

Jonathan Vernon
10:02 on 23 January 2013

 

Fig.1. A game of pairs - our minds are far more interprative, chaotic and illogical when it comes to visual associations based on what we see around us.

Automatically Augmenting Lifelog Events Using Pervasively Generated Content from Millions of People

In the pursuit of pervasive user-generated content (ugc) based on senors, by augmenting visual lifelogs with 'Web 2.0' content collected by millions of other individuals. 

We present a system that realises the aim of using visual content and sensor readings passively captured by an individual and then augmenting that with web content collected by other individuals. Doherty and Smeaton (2009)

  • Lifelogging, like keeping a diary, is a private and exclusive form of reverse surveillance. Doherty and Smeaton (2009)
  • Using SenseCam from Microsoft. Zacks (2006)
  • human memory operates by associating linked items together. Baddley (2004)
  • supportive of those patients suffering from early stage memory impairment. Berry e al (2009)
  • enhancing SenseCam gathered images by data mining from ugc sites such as Flickr and YouTube. Doherty and Smeaton (2009)
  • See also MyLifeBits. Bell and Gemmel (2007)
  • A commercial lifelogging product ViconRevue. OMG
  • Flickr has over 95 million geo-tagged images. (2010)
  • YouTube has 100 million video views per day (2010) YouTube fact_sheet

It has a camera and a range of other sensors for monitoring the wearer’s environment by detecting movement, temperature, light intensity, and the possible presence of other people in front of the device via body heat.

(I'd like the sensecam to be smaller still and include a microchip in a swimmer's cap, or goggle or swimsuit to monitor various other factors, including heart rate, blood sugar levels and carbon dioxide). 

How the mind disects, stores and correlates the information if gathers is somewhat different to the linear recording or cataloguing of current systems though. 

After her first stroke a patient found engagement when otherwise unable to communicate by looking at family photographs on an iPad. After a second stroke the same patient, deemed incapable of comprehension or communication, responded to hundreds of images of paintings she had known in her lifetime - in particular responding to the question posed when looking at one painting. Where is it? Ans; Louvre. What is it? Mona Lisa. (Vernon, 2012)

As sensing technologies become more ubiquitous and wearable a new trend of lifelogging and passive image capture is starting to take place and early clinical studies have shown much promise in aiding human memory. Doherty and Smeaton (2009)

However, it is presumptious, prescritpive and even manipulative to assume that a person recalls 'more of the same' when visualing sensing or surveying a place. The foibles of the human mind and system is that noises and smell, the temperature and weather, and the time of day have a part to play. I visit Trafalgar Square and smell pigeons even though they are long gone. I visit Buckingham Palace and recall finding a woman dead on the pavement one late evening. I see snow and think of the broken leg I got from skiing in my teens - not snowmen. I see any icecream van and think specifically of Beadnell Bay, Northumberland. 

The mind is far, far more complex than a fancy game of 'pairs'. I have perhaps 30,000 of my own images online, so why support, replace or supplement these with those taken by others? What if during my lifetime I tag, link and assocaite these images? How might these be linked to another personal log - a diary of some 2.5 million written over a 30 year period?

There are research challenges involved in further improving the quality of the lifelog augmentation process, especially with regard to “event-specific” lifelog events, e.g., football matches, rock concerts, etc. Other research challenges include investigations into selecting initial seed images based on adaptive radii, more sophisticated tag selection techniques, and also considering how interface design and varying methods of visualisation affect users’ acceptance of augmented data.

REFERENCE

Baddeley, A., Ed. Your Memory: A User’s Guide; Carlton Books: New York, NY, USA, 2004.

Bell, G.; Gemmell, J. A digital life. Scientific American Magazine, March 2007.

Berry, E.; Hampshire, A.; Rowe, J.; Hodges, S.; Kapur, N.; Watson, P.; Smyth, G.B.G.; Wood, K.;
Owen, A.M. The neural basis of effective memory therapy in a patient with limbic encephalitis. J.
Neurol. Neurosurg. Psychiatry 2009, 80, 582–601.

Doherty,.R. and Smeaton.A.F. (2009) Automatically Augmenting Lifelog Events Using Pervasively Generated Content from Millions of People

Vernon, J.F. (2012) Use of hundreds of image grabs of contempary artists, Leonardo da Vinci and Van Gogh to communicate with an elderly patient after a series of catastrophic and ultimately fatal strokes. 

Zacks, J.M.; Speer, N.K.; Vettel, J.M.; Jacoby, L.L. Event understanding and memory in healthy
aging and dementia of the alzheimer type. Psychol. Aging 2006, 21, 466–482.

 

Jonathan Vernon
11:55 on 23 January 2013

An Investigation into Automatically Captured Autobiographical Metadata, and the Support for Autobiographical Narrative Generation by Mischa M Tuffield - September 29, 2006

 

 

Jonathan Vernon
11:51 on 24 January 2013

Embedded Content

Contribute

Derek Chirnside
10:54pm 13 January 2013


You may be interested in Larry's dream:

http://cloudworks.ac.uk/cloud/view/7045

For me, I use a notebook and a smartphone. 

-Derek

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.