iOS AVPlayer – Localize your Videos/Audio

You would like to build an iPhone app that has many videos but you don't want to have 1 video per language.  This will take up a lot of disk space!  One solution is  to have 1 video and n audios (1 per language).  This allows you to compose the PlayerView of the video and the language audio the user selected.

Before we get into the AVFoundation framework, there are few steps you need to take:

1.  The Videos (without the audio) need to be created.

2.  Create the Audio files - 1 per language

3.  Localize Audio Files

Add Audio file to Xcode

Select Audio file and localize

Select Utilities View on top right

Select localization

Add languages you require

Copy the language audio to the appropriate directory   e.g.  es.lproj directory will be for the Spanish Audios

Say Hello to AVFoundation

PlayerViewController (UIViewController) will contain the PlayerView (UIView).  The PlayerView will contain the AVPlayer.

The classes we will be discussing are the AVPlayer, AVPlayerItem, AVMutableCompositionTrack, and AVAssetTrack.  These classes will help us put together a Video and Audio.

1.  Load the AVURLAsset

2.  Create AVMutableCompostion - allows you to create a composition from existing assets.  We create the AVMutableCompositionTrack for the Video.  Next will create the AVAssetTrack for the Video and insert it into the composition video track.  See code below.

3.  Load Audio Asset for the app language.  See code below.

To summarize the main "container" is the _languageVideoComposition (AVMutableComposition).  We use the _languageVideoComposition to create an AVMutableCompositoinTrack (one for AVMediaTypeVideo and one for AVMediaTypeAudio).  Next, the AVAssetTrack for the video and for the audio is created.  Each AVAssetTrack is set on the corresponding AVMutableCompositionTrack.

The AVPlayerItem is created with the _languageVideoComposition.  The AVPlayer is created using the AVPlayerItem.  Finally the playerView is set with the AVPlayer.

The sample code can be found here: iOS AVPlayer - Localize your videos and audios sample

Comment

Simple Ruby Monitoring and Management Interface For beanstalkd, beanstalkd_view

Beanstalk is great option if you need a fast lightweight message queue. The protocol is simple, and there are lots of examples and libraries online. (Here's a nice summary article.)

However, it does not come with a standard management UI.

Resque, a message queue built on top of Redis, has a great web management interface as part of its source base. It was implemented as a Sinatra app, that can even be embedded into a Rails app.

(See the Resque Railscast)

Being jealous of that interface, I decided to create a similar Sinatra app that can be used to manage and monitor your beanstalkd queues. The application is called beanstalkd_view, and the source code is available on github: https://github.com/denniskuczynski/beanstalkd_view

Configuration

Since beanstalkd_view is a Sinatra app it can be embedded into a Rails app. Simply mount the application in your routes.rb file:

Alternately, beanstalkd_view uses the Vegas gem, so you can start the application from the command line on an available port. Just execute the beanstalkd_view executable which will be available after executing gem install beanstalkd_view.

To configure beanstalkd_view, you have to set the ENV['BEANSTALK_URL'] with the URLs of your beanstalkd daemons.

  • If mounted in a Rails app, setting this variable could be done within an initializer.
  • If running from the command line, you'll currently have to edit the bin file: beanstalkd_view/bin/beanstalkd_view, which is hardcoded to the default localhost address of beanstalkd.

Features

beanstalkd_view is a simple two page application.

The index page shows the overall beanstalkd statistics: charts showing the counts of the different job types submitted and buried, and a basic form for submitting new jobs with JSON data.

A page providing specific tube statistics and actions can be accessed by clicking on a tube name at the top of the index page. This page shows the individual tube's statistics and also forms for kicking the tube, pausing the tube, and peeking at jobs.

Screenshot of beanstalkd_view

Behavior with multiple beanstalk queues

Beanstalkd_view leverages the beanstalk-client gem to interact with the beanstalkd daemons. This library is great because it provides a ConnectionPool object to manage interacting with multiple beanstalkd daemons.

(Beanstalkd does not natively support any form of clustering, so multiple instances must be managed by the client.)

When using multiple queues:

Actions Behavior
Queue statistic, tube listing, and pause commands Sent to all queue instances
Kick and add new job commands (put) Sent to random queue instances
Peek and delete job commands (put) Sent to each queue instance, looking for response

Feedback

See the github page for more information.

Please send back any feedback if you find beanstalkd_view useful. Or submit pull requests on github.

Comments

Saving Rotated Images on the iPhone

Rotating the image on the screen is very simple, but re-saving rotated images can be a little tricky. If you are sending photos over the network or saving photos to the iPhone's camera roll, then you need to know how to get them to be oriented correctly.

There are two basic ways to rotate a JPEG/PNG:

  1. Rotate the actual data in the file
  2. Set the EXIF data flag for the rotation you need
Rotating the data means going through each pixel and copying the pixels to another buffer and then re-compressing the image, which is slow and will reduce the quality of the image. Changing the EXIF tag is very fast and doesn't damage the quality of the image, but some software ignores or strips the EXIF tag and will display the image incorrectly (e.g. when you upload an image to Facebook).

Rotating with EXIF tag

Even if you always want to transform the actual data in your file you need to know about EXIF orientations. The iPhone's built-in camera saves pictures as JPEGs with the orientation stored as an EXIF tag. If you upload these or try to re-transform them you need to take the EXIF data into account.
UIImage has rather strange support for EXIF rotation. If you load a JPEG with EXIF orientation, then the orientation is stored in the UIImage's imageOrientation property and UIKit uses this to properly display the image. If you load a PNG, then the imageOrientation is always in its default "up" orientation, but the imageData attached to the PNG is rotated for you. If you save a UIImage with UIImageJPEGRepresentation(), then the imageOrientation property is used in the file data to create an EXIF tag. If you save a UIImage with UIImagePNGRepresentation(), then the imageOrientation is not saved and the image data is not rotated, so the image may not display the same if you reopen it.
Format Save Load Save Function
JPEG EXIF loaded into imageOrientation EXIF saved from imageOrientation UIImageJPEGRepresentation()
PNG imageOrientation lost EXIF used to transform data automagically UIImagePNGRepresentation()
To rotate a UIImage 90°, the easiest way is to simply generate a new UIImage with the same CGImage data backing it, but a new orientation.

The newImage will retain the CGImage and everything is hunky dory. You can save this as a JPEG or to the camera roll and all will be good in your world (as long as you don't upload to Facebook).

Rotating the actual data

If you are only dealing with a small image then just rotating it on the main thread is easy (This should work on all iOS2+ devices)

Apple has made some of UIKit thread safe in iOS4, but they haven't decided to tell us what is thread safe... If you have a Apple Developer Account check out this devforum thread where an Apple engineer explains that the above code IS thread safe on devices running iOS4+. As usual, if something isn't explicitly stated as thread safe then you should assume the worst. From what I've gathered from various sources the following should be true:
  1. imageNamed: is not thread safe as of iOS4.3
  2. imageWithContentsWithFile: is thread safe in iOS4.0, but a bug makes it not load the high res @2x versions of files. The bug was fixed in iOS4.1.
  3. All of the drawing methods for UIKit should be thread safe, but I'd recommend filing a bug report with Apple listing the methods you want to use if they are not documented as such. The bug I filed is titled "UIGraphicsBeginImageContextWithOptions is documented incorrectly". I'll update this post whenever Apple gets to my ticket
A more explicit way to rotate an image on another thread is to use CoreGraphics. If you use Core Graphics, you must also drop down to using CGImages. CGImages do not have an orientation property, so you must figure out the correct orientations yourself. A thread safe Core Graphics version that uses GCD would be:
Depending on your application's requirements you may use any one or all of the above methods for rotating an image. EXIF orientations are extremely fast but not always supported. UIGraphic/UIKit rotations are simple to write, but aren't documented as being thread safe. Core Graphics rotations are the most difficult, but they are the documented way to do it in a thread safe way.
Comments

A UIScrollView You Can Be Proud Of

Ever wondered how Apple created the slider used to display an app's screenshots? While it is a UIScrollView at heart, most developers know that by design a UIScrollView with paging enabled only stops at multiples of its frames width (or height). Getting those overhanging views on the side is no easy task but with a bit of code I'll show you how it can be done.

The first step is to figure out how wide you want your pages to be. Make that the width of the UIScrollView, then set your subviews sizes accordingly, making sure their centers are based on multiples of the UIScrollViews width. If you run your project now you'll notice that all you see is the center view since it causes the subviews to be clipped by the bounds of the receiver. So, we're going to add a line of code to change this:

[scrollView_ setClipsToBounds:NO];

Now the subviews whose frames extend beyond the visible bounds of the receiver aren't clipped anymore. But this introduces another problem; the UIScrollView won't scroll if  user tries to drag from anywhere outside the range of its frame. What we need to do now is create a UIView subclass that will contain the UIScrollView and its subviews. Fundamentally, it should have the frame of what we're expecting the UIScrollView to have under normal conditions. Place the UIScrollView in the center of this view and make sure the UIView's clipsToBounds is set to YES (the default is NO).

The final step is to override the - (UIView*)hitTest:WithEvent: inside the UIView:

- (UIView*)hitTest:(CGPoint)point withEvent:(UIEvent *)event {
  if ([self pointInside:point withEvent:event]) {
    return scrollView_;
  }
  return nil;
}

And there you have it! Now if you try to scroll anywhere within the frame of the UIView you created, the UIScrollView will respond to those actions.

Now to make things even more complicated, we're going to display  a UIActionSheet whenever a user taps on the center view. You can simply add a UITapGestureRecognizer to the UIView but you'll also notice that the UIActionSheet will come up even if the user taps one of the adjacent subviews in the UIScrollView. This is just bad usability.

Instead, we're going to create a UIScrollView subclass just so we can override its touchesBegan and touchesEnded methods. In touchesBegan, we're first going to determine if the user tapped within the bounds of the center view and keep an NSTimeInterval of it:

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
  CGRect b = [self bounds];
  CGFloat x = CGRectGetMaxX(b) - gallerySize__.width;
  CGRect placeholder = CGRectMake(x, b.size.height/2 - gallerySize__.height/2, gallerySize__.width, gallerySize__.height);

  UITouch* touch = [touches anyObject];
  CGPoint touchPoint = [touch locationInView:self];
  if (CGRectContainsPoint(placeholder, touchPoint)) {
    NSInteger numTaps = [[touches anyObject] tapCount];
    if (numTaps == 1) {
      startTime_ = [NSDate timeIntervalSinceReferenceDate];
    }
  }
}

Then in the touchesEnded method, we're going to fire off a post notification if the user keeps their finger there before a set time we feel comfortable with (I've found .5 seconds to work well). We do this to make sure the user is tapping as opposed to a swipe :

- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
  if (startTime_) {
    NSTimeInterval stopTime = [NSDate timeIntervalSinceReferenceDate];
    CGFloat holdTime = stopTime - startTime_;
    if (holdTime < .50) {
      [[NSNotificationCenter defaultCenter] postNotificationName:@"TAPPED_INSIDE_CUSTOM_SCROLLVIEW" object:nil];
    }
  }
}

And thats it! Now you can go show off to your friends that you can program just like the guys at Apple. I've also taken the liberty to add some convenience methods for setting the background image, gallery size, and spacer size. Additionally you'll find two optional delegates there. One for when the user finishes swiping, and the other for tapping on the center view, as illustrated above. Feel free to add your own, and be sure to show us what you've got. We'd love to see it!

You can find a video demo of this project along with its source code here.

Comments

Speed Up Maven GWT Compile Time For Developers

Google Web Toolkit is great for when you need to do AJAX on a Java centric web application. When you want to ensure type safety from the AJAX web layer all the way through to your persistence layer, there is nothing better than GWT.

Unfortunately because GWT code has to support 5 different rendering engines in the current major browsers, compilation time can be rather slow.  Our current project has six different GWT modules and the GWT compile step was starting to impact the code, compile, test, fix, rinse repeat cycle of our developers.

Add on top of this each locale supported by the app and the compilation time can start to get really long as GWT has to compile each module once for each locale and rendering engine combination.

Ideally you would use GWT Hosted mode for your development cycle, but sometimes you just need to run it as compiled JavaScript, thus requiring a lengthy recompile of your GWT modules.

The GWT docs actually have a section on how to speed up compile time by having the developers run the compile for one browser only.  However this involves creating an extra GWT module for each one of your existing modules.  We have five and creating five more modules to implement this speed up was rather unappealing.  There had to be a better way.

We are using Maven to manage our build process.  Surely there has to be a way to utilize Maven's very flexible configuration and plugin architecture to let us specify a profile that builds the GWT modules for only one browser rendering engine.

Here is the solution I came up with.

The key to compiling GWT for just one browser is in the user.agent property as shown in the GWT docs mentioned above.

The GWT docs show setting this property in the GWT module file for the browser specific HelloFirefox child module that inherits from the Hello module.

What we want is to have a parent module set the user.agent property and have all of our modules inherit from this module.  This property then needs to be set by Maven depending on which build profile we specify.

To this end, I created a MavenFilteredUserAgent module.  But because I need the property to be set by a Maven profile I had to work some Maven magic.

The trick here is to get Maven to filter the module file and then get GWT to use the filtered version during compile.

Here is the Maven magic that makes all this possible:

pom.xml

MavenFilteredUserAgent.gwt.xml

MyCompileOptimizedGWTModule.gwt.xml

Have all of your GWT modules inherit the MavenFilteredUserAgent as in the MyCompileOptimizedGWTModule example above.

Once in place you can then run

mvn install -Pgwt-firefox

and all of your GWT modules will compile for Firefox only.

You can now test your GWT app in Firefox having spent considerably less time waiting for it to compile.

Be very careful when using this speed up technique that you don't run your GWT modules in a browser different from the one you compiled for as you will get very confusing and meaningless JavaScript errors.  And definitely make sure that you never deploy an app compiled for just one browser to production.  GWT has been improved to make this mismatch more obvious as mentioned in issue 5861, but this improvement won't be available till the next release of GWT.

See Also:

Comment