News

iOS AVPlayer – Localize your Videos/Audio

You would like to build an iPhone app that has many videos but you don't want to have 1 video per language.  This will take up a lot of disk space!  One solution is  to have 1 video and n audios (1 per language).  This allows you to compose the PlayerView of the video and the language audio the user selected.

Before we get into the AVFoundation framework, there are few steps you need to take:

1.  The Videos (without the audio) need to be created.

2.  Create the Audio files - 1 per language

3.  Localize Audio Files

Add Audio file to Xcode

Select Audio file and localize

Select Utilities View on top right

Select localization

Add languages you require

Copy the language audio to the appropriate directory   e.g.  es.lproj directory will be for the Spanish Audios

Say Hello to AVFoundation

PlayerViewController (UIViewController) will contain the PlayerView (UIView).  The PlayerView will contain the AVPlayer.

The classes we will be discussing are the AVPlayer, AVPlayerItem, AVMutableCompositionTrack, and AVAssetTrack.  These classes will help us put together a Video and Audio.

1.  Load the AVURLAsset

2.  Create AVMutableCompostion - allows you to create a composition from existing assets.  We create the AVMutableCompositionTrack for the Video.  Next will create the AVAssetTrack for the Video and insert it into the composition video track.  See code below.

3.  Load Audio Asset for the app language.  See code below.

To summarize the main "container" is the _languageVideoComposition (AVMutableComposition).  We use the _languageVideoComposition to create an AVMutableCompositoinTrack (one for AVMediaTypeVideo and one for AVMediaTypeAudio).  Next, the AVAssetTrack for the video and for the audio is created.  Each AVAssetTrack is set on the corresponding AVMutableCompositionTrack.

The AVPlayerItem is created with the _languageVideoComposition.  The AVPlayer is created using the AVPlayerItem.  Finally the playerView is set with the AVPlayer.

The sample code can be found here: iOS AVPlayer - Localize your videos and audios sample

Comment

Simple Ruby Monitoring and Management Interface For beanstalkd, beanstalkd_view

Beanstalk is great option if you need a fast lightweight message queue. The protocol is simple, and there are lots of examples and libraries online. (Here's a nice summary article.)

However, it does not come with a standard management UI.

Resque, a message queue built on top of Redis, has a great web management interface as part of its source base. It was implemented as a Sinatra app, that can even be embedded into a Rails app.

(See the Resque Railscast)

Being jealous of that interface, I decided to create a similar Sinatra app that can be used to manage and monitor your beanstalkd queues. The application is called beanstalkd_view, and the source code is available on github: https://github.com/denniskuczynski/beanstalkd_view

Configuration

Since beanstalkd_view is a Sinatra app it can be embedded into a Rails app. Simply mount the application in your routes.rb file:

Alternately, beanstalkd_view uses the Vegas gem, so you can start the application from the command line on an available port. Just execute the beanstalkd_view executable which will be available after executing gem install beanstalkd_view.

To configure beanstalkd_view, you have to set the ENV['BEANSTALK_URL'] with the URLs of your beanstalkd daemons.

  • If mounted in a Rails app, setting this variable could be done within an initializer.
  • If running from the command line, you'll currently have to edit the bin file: beanstalkd_view/bin/beanstalkd_view, which is hardcoded to the default localhost address of beanstalkd.

Features

beanstalkd_view is a simple two page application.

The index page shows the overall beanstalkd statistics: charts showing the counts of the different job types submitted and buried, and a basic form for submitting new jobs with JSON data.

A page providing specific tube statistics and actions can be accessed by clicking on a tube name at the top of the index page. This page shows the individual tube's statistics and also forms for kicking the tube, pausing the tube, and peeking at jobs.

Screenshot of beanstalkd_view

Behavior with multiple beanstalk queues

Beanstalkd_view leverages the beanstalk-client gem to interact with the beanstalkd daemons. This library is great because it provides a ConnectionPool object to manage interacting with multiple beanstalkd daemons.

(Beanstalkd does not natively support any form of clustering, so multiple instances must be managed by the client.)

When using multiple queues:

Actions Behavior
Queue statistic, tube listing, and pause commands Sent to all queue instances
Kick and add new job commands (put) Sent to random queue instances
Peek and delete job commands (put) Sent to each queue instance, looking for response

Feedback

See the github page for more information.

Please send back any feedback if you find beanstalkd_view useful. Or submit pull requests on github.

Comments

Saving Rotated Images on the iPhone

Rotating the image on the screen is very simple, but re-saving rotated images can be a little tricky. If you are sending photos over the network or saving photos to the iPhone's camera roll, then you need to know how to get them to be oriented correctly.

There are two basic ways to rotate a JPEG/PNG:

  1. Rotate the actual data in the file
  2. Set the EXIF data flag for the rotation you need
Rotating the data means going through each pixel and copying the pixels to another buffer and then re-compressing the image, which is slow and will reduce the quality of the image. Changing the EXIF tag is very fast and doesn't damage the quality of the image, but some software ignores or strips the EXIF tag and will display the image incorrectly (e.g. when you upload an image to Facebook).

Rotating with EXIF tag

Even if you always want to transform the actual data in your file you need to know about EXIF orientations. The iPhone's built-in camera saves pictures as JPEGs with the orientation stored as an EXIF tag. If you upload these or try to re-transform them you need to take the EXIF data into account.
UIImage has rather strange support for EXIF rotation. If you load a JPEG with EXIF orientation, then the orientation is stored in the UIImage's imageOrientation property and UIKit uses this to properly display the image. If you load a PNG, then the imageOrientation is always in its default "up" orientation, but the imageData attached to the PNG is rotated for you. If you save a UIImage with UIImageJPEGRepresentation(), then the imageOrientation property is used in the file data to create an EXIF tag. If you save a UIImage with UIImagePNGRepresentation(), then the imageOrientation is not saved and the image data is not rotated, so the image may not display the same if you reopen it.
Format Save Load Save Function
JPEG EXIF loaded into imageOrientation EXIF saved from imageOrientation UIImageJPEGRepresentation()
PNG imageOrientation lost EXIF used to transform data automagically UIImagePNGRepresentation()
To rotate a UIImage 90°, the easiest way is to simply generate a new UIImage with the same CGImage data backing it, but a new orientation.

The newImage will retain the CGImage and everything is hunky dory. You can save this as a JPEG or to the camera roll and all will be good in your world (as long as you don't upload to Facebook).

Rotating the actual data

If you are only dealing with a small image then just rotating it on the main thread is easy (This should work on all iOS2+ devices)

Apple has made some of UIKit thread safe in iOS4, but they haven't decided to tell us what is thread safe... If you have a Apple Developer Account check out this devforum thread where an Apple engineer explains that the above code IS thread safe on devices running iOS4+. As usual, if something isn't explicitly stated as thread safe then you should assume the worst. From what I've gathered from various sources the following should be true:
  1. imageNamed: is not thread safe as of iOS4.3
  2. imageWithContentsWithFile: is thread safe in iOS4.0, but a bug makes it not load the high res @2x versions of files. The bug was fixed in iOS4.1.
  3. All of the drawing methods for UIKit should be thread safe, but I'd recommend filing a bug report with Apple listing the methods you want to use if they are not documented as such. The bug I filed is titled "UIGraphicsBeginImageContextWithOptions is documented incorrectly". I'll update this post whenever Apple gets to my ticket
A more explicit way to rotate an image on another thread is to use CoreGraphics. If you use Core Graphics, you must also drop down to using CGImages. CGImages do not have an orientation property, so you must figure out the correct orientations yourself. A thread safe Core Graphics version that uses GCD would be:
Depending on your application's requirements you may use any one or all of the above methods for rotating an image. EXIF orientations are extremely fast but not always supported. UIGraphic/UIKit rotations are simple to write, but aren't documented as being thread safe. Core Graphics rotations are the most difficult, but they are the documented way to do it in a thread safe way.
Comments

Speed Up Maven GWT Compile Time For Developers

Google Web Toolkit is great for when you need to do AJAX on a Java centric web application. When you want to ensure type safety from the AJAX web layer all the way through to your persistence layer, there is nothing better than GWT.

Unfortunately because GWT code has to support 5 different rendering engines in the current major browsers, compilation time can be rather slow.  Our current project has six different GWT modules and the GWT compile step was starting to impact the code, compile, test, fix, rinse repeat cycle of our developers.

Add on top of this each locale supported by the app and the compilation time can start to get really long as GWT has to compile each module once for each locale and rendering engine combination.

Ideally you would use GWT Hosted mode for your development cycle, but sometimes you just need to run it as compiled JavaScript, thus requiring a lengthy recompile of your GWT modules.

The GWT docs actually have a section on how to speed up compile time by having the developers run the compile for one browser only.  However this involves creating an extra GWT module for each one of your existing modules.  We have five and creating five more modules to implement this speed up was rather unappealing.  There had to be a better way.

We are using Maven to manage our build process.  Surely there has to be a way to utilize Maven's very flexible configuration and plugin architecture to let us specify a profile that builds the GWT modules for only one browser rendering engine.

Here is the solution I came up with.

The key to compiling GWT for just one browser is in the user.agent property as shown in the GWT docs mentioned above.

The GWT docs show setting this property in the GWT module file for the browser specific HelloFirefox child module that inherits from the Hello module.

What we want is to have a parent module set the user.agent property and have all of our modules inherit from this module.  This property then needs to be set by Maven depending on which build profile we specify.

To this end, I created a MavenFilteredUserAgent module.  But because I need the property to be set by a Maven profile I had to work some Maven magic.

The trick here is to get Maven to filter the module file and then get GWT to use the filtered version during compile.

Here is the Maven magic that makes all this possible:

pom.xml

MavenFilteredUserAgent.gwt.xml

MyCompileOptimizedGWTModule.gwt.xml

Have all of your GWT modules inherit the MavenFilteredUserAgent as in the MyCompileOptimizedGWTModule example above.

Once in place you can then run

mvn install -Pgwt-firefox

and all of your GWT modules will compile for Firefox only.

You can now test your GWT app in Firefox having spent considerably less time waiting for it to compile.

Be very careful when using this speed up technique that you don't run your GWT modules in a browser different from the one you compiled for as you will get very confusing and meaningless JavaScript errors.  And definitely make sure that you never deploy an app compiled for just one browser to production.  GWT has been improved to make this mismatch more obvious as mentioned in issue 5861, but this improvement won't be available till the next release of GWT.

See Also:

Comment

Smooth Incremental Slider for Flex

So I don't like the way the Flex 4 slider moves in increments when you have the snap interval set to something large. It means you have to drag it a certain distance before it moves at all, which could mislead users into thinking it isn't slideable . So I extended the component to make it drag smoothly, but when you release the slider it will jump to the correct step.

Here is the running example. The grey rectangle's width property is bound to my new slider property "jumpedValue":

So my component has two new properties: "jumpSize" determines the increment size and "jumpedValue" gives the snapped value. The latter is bindable and it is the main property other components should listen to to get the value of the slider. All I needed to do was listen to Event.CHANGE and then update the jumpedValue.

To ensure it jumps to the correct point once you release it, I overrode the system mouse up handler:

There was one other behavior I needed. When you click the track, I wanted the button to slide to the closest snap-point rather than to the point you clicked. The easiest way I could find to do that was to override the track click handler so it sets a private property 'jumpNow', and then override 'nearestValidValue' so that it snaps straight away if jumpNow is true:




And that's it. As with most Flex code, they key is not what you do but when you do it.

Comments