Exhibition learnings 5 – encoding videos for mobiles

This is the last post summing up our experiences from putting together the 3 Inch Canvas exhibition. This time we sum up what we learned about using videos on mobile devices.

If one looks at the second 3 Inch Canvas exhibition from the perspective of what technical aspects required the most effort, two things stand out. One is quite obvious: the work required to develop the exhibition applications. The other aspect is perhaps less obvious. It is the effort required to manage the video formats for the participating works.

The video experience created by services like YouTube or Vimeo or just the media players on PC’s and Mac’s give an illusion of effortlessness. However, the fact is that a lot of work have gone into these services and applications to hide a lot of the complexities of video processing.

Mobile phones unfortunately do not yet come with video players that are as versatile as those on desktops. And even if they did, there are still a few things that would be worth considering.

Video formats

The first thing you will face when trying to display videos on a mobile device are the limitations in terms of supported video formats. For information on video formats, see this excellent post before reading on.

The reason to bring up video formats is that not all format are supported on all mobile devices. From practical experience, Nokia devices seemed most the sensitive to the video format used. If a video played on a Nokia device, it would most likely work on Android and Windows Phone devices as well. We did not explicitly test on an iPhone.

You can use the video information for the X6 device, found here, as a guideline for creating videos that work on most mobile devices. In order to view videos at full X6 resolution (meaning 640×360, called nHD in the spec) you need to code the video using MPEG4 Simple profile. In video editing software this format is referred to by many terms, eg. just MPEG4 or MPEG4 Visual or MPEG4 Part 2.

The H.264 encoding that seems to be often automatically proposed by MPEG4 encoding software – referred to with terms such as H.264 or MPEG4 Part 10 or MPEG4 AVC – is not the same and is not supported by some Nokia devices such as the X6.

Video quality

The other aspect of video coding is the quality of the video. The video quality is decided mainly by two parameters: the video image size in pixels and the coding speed.

The video image (or display) size is a simple and straightforward concept. The more pixels used in coding the video, the better the video quality will potentially be. This is very similar to the effect of more pixels on still images.

For videos with an aspect ratio of 16:9, some of the most common pixel sizes are multiples of the fullHD video which has a pixel size of 1920 x 1080. The 16:9 formats are commonly referred to by the number of vertical pixels, hence the fullHD format can be referred to as 1080. Other common sizes are eg.

• 360 x 640 (referred to as 360)
• 480 x 854 (referred to as 480)
• 720 x 1280 (referred to as 720)

The other factor is the coding speed. This parameter indicates how many bit/second of data is used to code the video. Advanced coders tend to use an encoding referred to as variable bit rate. What this means is that the information (in this case video) is coded with a fixed quality and the average bit rate over a suitable time period is given. See this for details.

If we compare to jpeg images, the coding speed is a bit like the “file size” of jpeg images. The bigger the file (for a given pixel size), the better the image quality.

Coding for mobiles – selecting the appropriate video image size and coding speed

The video image size and the coding speed are connected. The coding speed is a measure of how much information is “consumed” to code the video. More information results in better quality until some level when increasing the coding speed no longer improves the quality.

The bigger the video image size, the more information is required to code the video at a given quality level. Thus a fullHD video (with a 1920×1080 display size) will need a higher bit rate than a small 640×360 video.

Why is this interesting? It is interesting from the perspective of how big a file or how much network bandwidth is needed to show a video. If we use a fullHD vide0, the bit rate might be 10Mbit/s. This means that a one minute video will need about  75Mbits of disk space. The smaller video can be coded at 1/10th of the bit rate, meaning about 1Mbit/s. In this case the resulting file will be just 7.5 Mbytes in size.

Where this matters the most is when a video is streamed over a network connection. Streaming means that the video is stored on one system but is being shown on an other system. The transfer of the video between the systems happens “in the background” at the same time as the video is being shown.

The fullHD video encoded at 10Mbit/s will need a 10Mbit/s network connection to be streamed. If the network bandwidth is lower, then the network will not be able to “top up” the device on which the video will be shown fast enough and the video will pause. This results in a very unpleasant viewing experience.

If we were to try to stream the video from a server to a mobile device, 10Mbit/s means that this can only be done over a wireless LAN (WLAN) connecting. The bandwidth of the cellular network is not (yet) capable of 10Mbit/s.

If we wanted to show the video on the mobile device we notice that fullHD (1980×1080) is actually unnecessarily big, mobile devices do not have display with that kind of resolution. We can make the video “smaller” by converting it to a more suitable size, let’s say 360 x 640.

Now the coding speed comes into play. If we just reduce the bitsize to 360 x 640 but do not lower the coding speed, we will end up with a video that still needs a 10Mbit/s bandwidth. 

We need to also reduce the coding speed to something more appropriate, in this case 1Mbit/second. A video that is encoded at 1Mbit/second just might be able to make it over a cellular network.

We can also make the video even smaller. In this case the video will not fill the entire screen of an advanced mobile phone. We can show the video as a small “postcard” on the mobile or we can enlarge the video on the device to fill be full screen. In both cases, the end result will be a less than perfect video viewing experience.

Finding the optimal encoding speed and image size is a tricky issue where trying out different combinations is the key to success. 

The 3 Inch Canvas videos

For the 3 Inch Canvas exhibition, the videos were encoded into 3 different formats.

• MPEG4 at the nonstandard size of 800 x 450. This format was used for the videos that were shown on the Windows Phone devices. The video image size was selected because the Windows Phone screen is 800 pixels wide. 800 x 450 is therefore the maximal video size for a video with and aspect ratio of 16:9 that can be shown on the device without being scaled either up or down.  Experimenting with the coding speed showed that a speed on about 1.5Mbit/second yielded an optimal video quality, increasing the coding speed did not improve visibly the video quality.
• MPEG4 at the nHD size 640 x 360. This video format was used on the non-interactive works that used Nokia X6 devices. It was also the format in which the videos were submitted to Arena Interactive for the Artwork of the day service.  A coding speed of about 0.9 – 1Mbit/second was found to be appropriate.
• HD or fullHD videos at 1280 x 720 or 1980 x 1080. This format was used for streaming videos from the Windows Phone devices to the big screens (see this post for more information). The video coding speed of about 5 Mbit/sec for HD and 10 Mbit/s for fullHD was found to yield good results.

The exhibition participants were requested to encode the videos themselves. The main thinking here was that the artists in this way would be able to control the quality of their videos.

There were no issues with the HD ro fullHD videos submitted by the artists. This comes as no surprise, as this is the format in which artists would create their videos to be shown on PCs.

For the smaller formats, there we quite a few issues, which resulted in the need to ask artist to recode and resend their submissions. Some of the common issues were:

• MPEG4 (or MPEG Simple profile) videos were very often encoded and sent as H.264 videos. This is in part our fault, as we did not realize up from how many terms were used for the two formats in various software packages and especially their presets.
• Inappropriate coding speed. Often the coding speed for the smaller videos was set to be very high. There were no cases of too low coding speed.

Designing for the small screen

Not directly connected to video encoding, but very much related to the overall video quality is the need to take the small physical screen size into account.

It was quite obvious that some of the videos had not been tested on small screens. This could mainly be seen in texts shown in the video such as credits at the end or texts that were part of the work itself.

When a video is being edited on a big screen, even small text is very easily readable. However, when the video is being shown on a small screen, the small text can become totally unreadable. Even if a mobile phone is not used during editing, it is still possible to test the video by shrinking the video preview window to the same size as a mobile phone screen. If the text is readable in the shrunk window, it will also be readable on the mobile screen.

An other thing to pay attention to is reading speed. If the text is quite small, users tend to take a bit longer to read it. Text that is scrolling or long text that is supposed to be read, needs to be displayed long enough so that users also on mobile devices have time to read it.

Conclusions

The main conclusion about using videos in mobile phone exhibitions is that good preparation on behalf of the organizer is required. The organizer should know up front what video formats are needed and provide detailed instructions to participants on how the participating videos should be encoded.

The organizer can of course do all the coding work (this is what we did in the first 3 Inch Canvas exhibition) but that is a lot of work and also makes the organizer ultimately responsible for the video quality.

3 Inch Canvas exhibition documentary on M2HZ

M2HZ is a web-based community TV-station operating in Helsinki. Emmi Vainio from M2Hz visited the 3 Inch Canvas exhibition when we were building it and did a great interview/short documentary about Live Herring ’11 and the 3 Inch Canvas concept.

Check out the video below. Finnish only, sorry, but you get a very nice glimpse of the “making of” even if you don’t speak the language! Thank you Emmi for a great presentation of what the 3 Inch Canvas is all about.

Click on the image below to see the video.

Exhibition learnings 4 – "Artwork of the day" service

The 3 Inch Canvas site visitor map by Google Analytics

 The “Artwork of the day” service was perhaps the most interesting thing we tried at the 2011 exhibition. The concept of the service was simple: each day during the Jyväskylä exhibition (and actually a few days after it closed), we released one new art video. Customers with advanced mobile phones could order the video using SMS at the modest cost of 0.95 €. After ordering, the customer could then watch the video on his/her mobile phone. 

There were a number of reasons for us to try this out. One reason was to extend the geographic reach of the exhibition. With the service we could make at least part of the exhibition accessible also to people who did not have the opportunity to visit the physical venue. 

The other reason was that the 3IC believes that art as a business and marketing of art and artists has its place in the digital, networked world. One of the purposes of the trial was to see if anyone else agreed with us. 

The outcome of the trial was kind of mixed. The artists were very positive to the trial and most artists with videos that we well suited for a wider distribution did participate. In this respect the trial was a success. 

The customer reaction to the service was not exactly overwhelming. In the end we got about 20 downloads. On the surface this number looks very, very low. However, if we put the number in perspective, things look a little brighter. During March, our site had some 500 unique visitors, of which a little over 400 were from Finland (numbers based on Google Analytics). Being from Finland is relevant as the service was restricted to the Finnish market.

Now, 20 downloads out of 400 visitors would correspond to about a 5 % conversion rate. And a 5 % conversion rate is excellent in the web world, in particular considering that the user needed to potentially change medium from a pc to a mobile phone to complete the purchase.  

Of course some of the artist did some marketing of their own and probably accounted for a rather large proportion of the downloads. Still, from a conversion rate perspective, the trial can be seen as a success. 

From a business perspective, the service obviously was a failure. One could argue that the main reason for this failure was the (low) number of people we reached with our marketing. If we could maintain even a fraction of the conversion rate and would succeed better with our marketing, then perhaps??

Lastly, a warm “Thank you” to Arena Interactive for creating the service. It speaks loads about the company that they ventured in artland and that we did not have a single customer complaint about anything related to the service!

Exhibition learnings 3 – showing videos from mobiles to big screens

The work "Keho" by Laura Rytkönen streamed to the big screen on the right.

One of the new things we tried in the Jyväskylä exhibition was to enable visitors to view the video works either on the mobile phones (Windows Phone 7) or on big TV monitors hung on the walls. According to the museum staff, the option to view the videos on a big screen turned out to be very popular. 

To show the videos stored on the phone on a big TV screen we used a technology called (nerd alert) DLNA. DLNA comes from the term Digital Living Network Alliance which does not say a lot. However, the main point of DLNA is that it is a specification for how devices can connect using a (wireless) network and share various media such as music, videos and photos. 

DLNA is not a standard feature of the Windows Phone 7 software, but for the exhibition we used LG devices and LG has included special DLNA software on their Windows Phone 7 models. The software (in technical terms a DLNA Mobile Digital Media Server) allows the LG phones to send media stored on the phones to DLNA compliant devices. 

What devices can then show media using DLNA? There are already quite a few devices out there that use DLNA, you can find a full list of devices here. Ideally we would have wanted to use a DLNA compliant TV displays at the exhibition, but unfortunately that was not possible. 

Instead we used Windows 7 PCs. The media player on Windows 7 is DLNA compliant. You can easily set it up to receive media from the network. Thus we hooked up 4 PCs to 4 big screens. On the phones we could send the videos to one of the PCs and then onto the big screens. 

How did this work? We were actually positively surprised at how well everything worked. We used different versions of the videos on the phone and to show on the big screen. The videos for the big screen were HD or full-HD versions (up to 1080×1920 pixels). Even the full HD videos worked in real time without any glitches! 

The only interesting detail with the setup was the Windows 7 PCs had to be connected to a wired network (and then to a WLAN hub) in order for the videos to stream. Streaming means that the video starts showing (almost) immediately. If the PC was connected directly to a wireless network, the videos did show up, but the Windows MediaPlayer first seemed to download the full video and only started to show it once the download had completed. 

Conclusion: Mobile DLNA really works!

Exhibition learnings 2 – embedding mobiles into works of art

Installation by Gun Holmström

This is the second post discussing some of the learnings from the second 3 Inch Canvas exhibition, see the first post here.   

One of the new things that we experimented with in the Jyväskylä exhibition was embedding mobile devices into something “larger”.   

A great example of such embedding was Gun Holmström’s installation “Chants for mobile phones” (shown left). In this work, videos are displayed on mobile phones and form part of the installation. It’s an interesting example of how mobile phones enable artist to embed digital content into their works also on other scales than by just using big computer screens or TVs. Hanging a big TV on one of the animals would not have worked too well.   

Embedding a mobile into an object, an installation or similar, enables the inanimate work to come alive and tell stories in totally new ways. In this exhibition we only used videos on the mobiles, but with programming the mobiles could be more interactive, possibly reacting to sounds, light and even movement.   

An other example is Denise Zieglers work “Autobus” (video below). Here the benefits of the small dimensions of the mobile are even more obvious as the mobile device creates an animated video backdrop for a toy car (about 5 cm/2 inches high).   

Based on the feedback from the artists, it seems that the exhibition acted as an eye-opener for some of the opportunities offered by mobile, just as we had hoped! 

Exhibition learnings – interactive artworks on mobiles

Visitor viewing the artwork on a mobile, the video screen in the background.

 The second 3 Inch Canvas exhibition for art on mobiles is now over and it is time to sum up our experiences and learnings. I’ll post a few notes with some of our observations. Hopefully these notes are useful for artists looking to use mobile devices to present their art and to curators that plan to use mobile devices as part of their exhibitions.  

Let’s first look at the more interactive part of the exhibition. The interactive part of the exhibition was presented on 9 mobile phones where the audience was supposed to pick up and handle the phones to gain access to the artwork. Each phone featured on artist and included a short presentation of the artist, a small introduction to the artist’s work and then the artworks themselves. The number of artworks varied between 1 and 3. The appearance of the applications can be seen in this post.  

The interactive applications were implemented using LG E-900 mobile phones using the Windows Mobile 7 operating system. Perhaps the most important technical feature of the devices was the big 3.8 inch display. Visitors all seemed to agree that it was possible to view and enjoy the artworks on the mobile phones with big displays. This confirms our similar experiences from last year’s exhibition.  

For video works (7 out of the 9 artworks), the visitor was also provided with the option to view the works on large TV-screens (see photo above). According to the museum staff, this turned out to be a very popular way to view the works. Unfortunately it was not possible to collect reliable statistical information comparing how often the works were viewed on the phone vs. how often they were viewed on the TV-screen, but from the staff accounts I suspect that viewing on the big screen was more popular.  

An other observation is that people still seemed quite shy to pick up and use the mobiles. Museum staff suggested that this might be because museums normally enforce a “do not touch” policy and that people therefore are a bit reluctant to touch things in a museum setting.  

It also seems that people simply are unused to this way of viewing art and therefore simply are not sure how to behave. When there was staff on hand breaking the first ice and showing the visitors how to operate the mobiles and how to view the works, then people would mostly go on to view also other works. 

From this we draw the conclusion that exhibitions that pick up new technologies and new ways of “consuming” art, will benefit from having staff on hand to encourage and teach visitors to use the artworks and the technical devices used to present the works.  

An other observation that might be made is that people seem to have a reasonably short span of attention when passively viewing artworks on mobile devices. The exhibition devices had some simple logging software integrated into the applications. Even though the numbers are far to small to make any reliable predictions about visitor behaviour, these logs enable us to calculate how often a video was viewed to the end (as opposed to interrupted in the middle). The information (shown in the graph below) gives some indication that the longer the work, the less likely it is that people will watch it to the end. This correlates with the experience from the first exhibition, even though last time we did collect any numerical information on this subject.  

  

It would be interesting to compare this behaviour to e.g. games. It seems that games and other applications can captivate people for longer periods of time. Is this due to the fact that art still puts the viewer in a rather “passive” role as simply the viewer while games and other applications activate the user more? Could media art in this case become more interesting if it provided more opportunities for the viewer to actively participate in the work?

Päivän teos 1.4 – Albert Merino – White rain

Yhdestoista ja viimeinen 3 Inch Canvas Päivän teos julkaistu! Tilaa teos lähettämällä alla oleva tekstiviesti numeroon 16323:

video 11

Tilauksen hinta on 0.95€, hinta veloitetaan matkapuhelinliittymäsi laskussa. Teos on tilattavissa 1.4. – 8.4. välisenä aikana. Teoksen kesto: 8 min 23 sek.

Tarkemmat tilausohjeet:

Palvelun käyttö vaati matkapuhelimen, jossa on dataliittymä (Internet-liittymä) sekä mahdollisuus näyttää videoita. Ennen tilaamista, varmista että matkapuhelimessasi on vaaditut ominaisuudet. Palvelun tarjoaja ei vastaa kuluista, jos teos on tilattu matkapuhelimeen jossa ei ole vaadittuja ominaisuuksia.

Videon siirto tehdään käyttäen puhelimen dataliittymää. Varmista siksi myös, että matkapuhelimessasi on suurien datamäärien siirtoon soveltua dataliittymä, mieluiten kiinteähintainen, rajattoman datamäärän liittymä. Voit myös katsoa teoksen WLANin kautta, jos puhelimessasi on WLAN-tuki.

Tilausviestissä ei saa olla ylimääräisiä välimerkkejä ennen sanaa ”video” eikä viestissä myöskään saa käyttää rivinvaihtoa.

Jos tilaus onnistuu, saat tekstiviestin, jossa on weblinkki tilaamaasi videon. Voit katsoa teokset klikkaamalla linkkiä. Jos tilaus ei jostain syystä onnistu, saat paluuviestinä virheilmoituksen. Tässä tapauksessa tilausta ei veloiteta.

Päivän teos 31.3 – Jaana Ristola – Sakka

Uusi 3 Inch Canvas Päivän teos julkaistu! Tilaa teos lähettämällä alla oleva tekstiviesti numeroon 16323:

video 10

Tilauksen hinta on 0.95€, hinta veloitetaan matkapuhelinliittymäsi laskussa. Teos on tilattavissa 31.3. – 8.4. välisenä aikana. Teoksen kesto: 2 min 54 sek.

Tarkemmat tilausohjeet:

Palvelun käyttö vaati matkapuhelimen, jossa on dataliittymä (Internet-liittymä) sekä mahdollisuus näyttää videoita. Ennen tilaamista, varmista että matkapuhelimessasi on vaaditut ominaisuudet. Palvelun tarjoaja ei vastaa kuluista, jos teos on tilattu matkapuhelimeen jossa ei ole vaadittuja ominaisuuksia.

Videon siirto tehdään käyttäen puhelimen dataliittymää. Varmista siksi myös, että matkapuhelimessasi on suurien datamäärien siirtoon soveltua dataliittymä, mieluiten kiinteähintainen, rajattoman datamäärän liittymä. Voit myös katsoa teoksen WLANin kautta, jos puhelimessasi on WLAN-tuki.

Tilausviestissä ei saa olla ylimääräisiä välimerkkejä ennen sanaa ”video” eikä viestissä myöskään saa käyttää rivinvaihtoa.

Jos tilaus onnistuu, saat tekstiviestin, jossa on weblinkki tilaamaasi videon. Voit katsoa teokset klikkaamalla linkkiä. Jos tilaus ei jostain syystä onnistu, saat paluuviestinä virheilmoituksen. Tässä tapauksessa tilausta ei veloiteta.

Päivän teos 30.3 – Laura Laukkanen – Leike

Uusi 3 Inch Canvas Päivän teos julkaistu! Tilaa teos lähettämällä alla oleva tekstiviesti numeroon 16323:

video 9

Tilauksen hinta on 0.95€, hinta veloitetaan matkapuhelinliittymäsi laskussa. Teos on tilattavissa 30.3. – 8.4. välisenä aikana. Teoksen kesto: 1 min 07 sek.

Tarkemmat tilausohjeet:

Palvelun käyttö vaati matkapuhelimen, jossa on dataliittymä (Internet-liittymä) sekä mahdollisuus näyttää videoita. Ennen tilaamista, varmista että matkapuhelimessasi on vaaditut ominaisuudet. Palvelun tarjoaja ei vastaa kuluista, jos teos on tilattu matkapuhelimeen jossa ei ole vaadittuja ominaisuuksia.

Videon siirto tehdään käyttäen puhelimen dataliittymää. Varmista siksi myös, että matkapuhelimessasi on suurien datamäärien siirtoon soveltua dataliittymä, mieluiten kiinteähintainen, rajattoman datamäärän liittymä. Voit myös katsoa teoksen WLANin kautta, jos puhelimessasi on WLAN-tuki.

Tilausviestissä ei saa olla ylimääräisiä välimerkkejä ennen sanaa ”video” eikä viestissä myöskään saa käyttää rivinvaihtoa.

Jos tilaus onnistuu, saat tekstiviestin, jossa on weblinkki tilaamaasi videon. Voit katsoa teokset klikkaamalla linkkiä. Jos tilaus ei jostain syystä onnistu, saat paluuviestinä virheilmoituksen. Tässä tapauksessa tilausta ei veloiteta.

Thank you Jyväskylä, we love you!

The museum part of the second 3 Inch Canvas exhibition has ended. According to the museum, just under 800 people visited the museum during the exhibtion! Compared to the roughly 100 visitors/week we had for the first exhibition, those numbers make us really happy. A big part of this is due to the fact that last Friday was Jyväskylä day with a lot of people all over the city. Great timing (which was no due to us).

It is now time to thank all those good people in Jyväskylä (and a few down “south” and “west”) who helped us create the exhibition.

First and foremost we want to thank all the artists!

You can find the full list of participants here!

Next in line is the Jyväskylä Art Museum: Thank you for having us.

Special thank to Jaana Oikari, curator (collections and archives) who handled the museum arrangements. A big hand also to technical supervisior Jarkko Kaunismäki and Jussi Reikki who built the exhibition!

Then, thanks to the Live Herring team.

The exhibition was part of the Live Herring ’11 event that spanned 6 weeks. This gargantuan project was handled by Päivi Hintsanen and Soile Ollikainen. Thanks also to Sirja Moberg who helped us with graphics design and took pictures of the event that also we have used.

The sponsors

Thanks to Microsoft Oy and Jukka Wallasvaara for loaning us the Windows Mobile 7 phones.

A big thanks also to Arena Interactive in Jyväskylä who set up the “Artwork of the day” service. In particular Partik Nygård and Jan Westin from the Arena Interactive Vaasa office who handled all the tech work related to the “Artwork of the day” -service.

(Remember, the “Artwork of the day” -service is still open until April 8, 2011.)

A few very helpful persons that we want to mention

Teemu Tapanila, from the Aalto University School of Science and Technology, for helping with the Windows Phone 7 programming.

Andreas Jakl, from Nokia, for helping with Qt programming for the Nokia devices used in the installation works.