肇鑫的技术博客

业精于勤，荒于嬉

Remove the video part from a live photo

Someone may think the process is as easy as getting the video from a live photo, removing the video and saving the other parts back. It is wrong.

We can get the video from a live photo using PHAssetResource's class func assetResources(for livePhoto: PHLivePhoto) -> [PHAssetResource]. But the PHAssetResource it gets contains empty assetLocalIdentifier. So you can't get its asset directly. And you can't remove it separately.

The correct way is to get the photo part, save it and remove the live photo.

Get the photo and save it

We could get the photo in three ways. Two from PHImageManager and one from PHAssetResourceManager. However, only one method is right way.

requestImage(for:targetSize:contentMode:options:resultHandler:)

OS, Mac Catalyst, tvOS
func requestImage(for asset: PHAsset, targetSize: CGSize, contentMode: PHImageContentMode, options: PHImageRequestOptions?, resultHandler: @escaping (UIImage?, [AnyHashable : Any]?) -> Void) -> PHImageRequestID
macOS
func requestImage(for asset: PHAsset, targetSize: CGSize, contentMode: PHImageContentMode, options: PHImageRequestOptions?, resultHandler: @escaping (NSImage?, [AnyHashable : Any]?) -> Void) -> PHImageRequestID

We should not use this method as it returns UIImage/NSImage, according to Apple, those two classes lacks metadata.

A UIImage object does not contain all metadata associated with the image file it was originally loaded from (for example, Exif tags such as geographic location, camera model, and exposure parameters). To ensure such metadata is saved in the Photos library, instead use the creationRequestForAssetFromImage(atFileURL:) method or the PHAssetCreationRequest class. To copy metadata from one file to another, see Image I/O.
creationRequestForAsset(from:)

requestData(for:options:dataReceivedHandler:completionHandler:)

We cannot use requestData(for:options:dataReceivedHandler:completionHandler:) of PHAssetResourceManager either. It does return Data instead of UIImage/NSImage. But the data it returns cannot save to PHPhotoLibrary correctly.

I think it is because of the data it returns contains the same localIdentifier as the live photo, which is allowed to save separately.

requestImageDataAndOrientation(for:options:resultHandler:)

This is the only way we should use to save the photo part from a live photo.

Remove the live photo

This is an easy job. Just use PHAssetChangeRequest.deleteAssets(_ assets: NSFastEnumeration).

Gain Insight of Serial and Concurrent Operations, Closure and async/await, Dispatch Semaphore and Dispatch Group

I understood those concepts deeper with a recent project.

Serial and Concurrent Operations

Serial

func serial() {
    for i in 0..<5 {
        let seconds:UInt32 = (1...3).randomElement()!
        sleep(seconds)
        print("\tWait \(seconds)s.")
        print(i)
    }
}

serial()

	Wait 3s.
0
	Wait 2s.
1
	Wait 1s.
2
	Wait 2s.
3
	Wait 1s.
4

Concurrent

func concurent() {
    let queue = DispatchQueue.global()
    
    for i in 0..<5 {
        queue.async {
            let seconds:UInt32 = (1...3).randomElement()!
            sleep(seconds)
            print("\tWait \(seconds)s.")
            print(i)
        }
    }
}

concurent()

	Wait 2s.
2
	Wait 3s.
0
	Wait 3s.
	Wait 3s.
	Wait 3s.
4
3
1

We can see that with serial operations, the codes are running one by one. And with concurrent operations, the codes are running in parallels.

Closure and async/await

Closure

func closure() {
    let url = URL(string: "https://zhaoxin.pro")!

    let task = URLSession.shared.dataTask(with: url) { data, urlResponse, error in
        if let error {
            print(error)
            return
        }
        
        if let httpResponse = urlResponse as? HTTPURLResponse,
           httpResponse.statusCode == 200,
           let data, let output = String(data: data, encoding: .utf8) {
            print(output)
        }
    }
    
    task.resume()
}

closure()

<!DOCTYPE html>
<!--[if IEMobile 7 ]><html class="no-js iem7"><![endif]-->
<!--[if lt IE 9]><html class="no-js lte-ie8"><![endif]-->
<!--[if (gt IE 8)|(gt IEMobile 7)|!(IEMobile)|!(IE)]><!--><html class="no-js"><!--<![endif]-->
<head>
  <meta charset="utf-8">
  <title>
  
  肇鑫的技术博客
  

  </title>
  <meta name="author" content="">
  <meta name="description" content="业精于勤，荒于嬉">
  ...

async/await

func async_await() async throws {
    let url = URL(string: "https://zhaoxin.pro")!
    let (data, urlResponse) = try await URLSession.shared.data(from: url)
    if let httpResponse = urlResponse as? HTTPURLResponse,
       httpResponse.statusCode == 200,
       let output = String(data: data, encoding: .utf8) {
        print(output)
    }
}

do {
    try await async_await()
} catch let error {
    print(error)
}

As you can see, most of the closures can convert to async/await, which makes them easy to understand. However, not all closures could be converted as async/await. For example, there are many API in Photos could not be converted to async/await. As those closures are called more than one time.

Dispatch Semaphore and Dispatch Group

For concurrent operations, if we want to do something after all operations are finished. We can use a timer, a Dispatch Semaphore, or a Dispatch Group.

Timer

func concurrentWithTimer() {
    let queue = DispatchQueue.global()
    let total = 5
    var finished = 0
    
    let lock = NSRecursiveLock()
    
    for i in 0..<5 {
        queue.async {
            let seconds:UInt32 = (1...3).randomElement()!
            sleep(seconds)
            print("\tWait \(seconds)s.")
            print(i)

            lock.lock()
            finished += 1
            lock.unlock()
        }
    }
    
    Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true, block: { timer in
        if finished == total {
            timer.invalidate()
            print("All Finished")
        }
    })
}

concurrentWithTimer()

	Wait 1s.
	Wait 1s.
0
2
	Wait 2s.
3
	Wait 3s.
	Wait 3s.
4
1
All Finished

Using Timer is easy to understand, but it costs more as it runs many times.

Dispatch Semaphore

func dispatch_semaphore() {
    let queue = DispatchQueue.global()
    let semaphore = DispatchSemaphore(value: 0)
    let total = 5
    var finished = 0
    
    let lock = NSRecursiveLock()
    
    for i in 0..<5 {
        queue.async {
            let seconds:UInt32 = (1...3).randomElement()!
            sleep(seconds)
            print("\tWait \(seconds)s.")
            print(i)

            lock.lock()
            
            finished += 1
            
            if finished == total {
                semaphore.signal()
            }
            
            lock.unlock()
        }
    }
    
    semaphore.wait()
    
    print("All Finished")
}

Semaphore only runs one time. But you need to deal with extra local variables, even more, you have to use lock to void data racing.

Dispatch Group

func dispatch_group() {
    let queue = DispatchQueue.global()
    let group = DispatchGroup()
    
    for i in 0..<5 {
        group.enter()
        
        queue.async {
            let seconds:UInt32 = (1...3).randomElement()!
            sleep(seconds)
            print("\tWait \(seconds)s.")
            print(i)

            group.leave()
        }
    }
    
    group.wait()
    
    print("All Finished")
}

Dispatch Group is the most elegant way to do the same job. There is no extra variables and no more extra operations. Just enter and leave then wait. Everything works like a charm.

VideoPlayer Turns Black Screen When Replace a Player Item

I was working on a project that shows videos from the Photo Library. However, each time I replaced the current video, the VideoPlayer turned black screen for a while.

I had no idea why this happened so I looked for what Photos app did for videos. It turned out that Photos showed a preview of the video first and then automatically played the video when the video data was loaded.

I did the same. First I fetched the preview image. When the preview was downloaded, I fetched the video.

The issue was the same. After some investigation, I found that the method of fetching video was called twice, which was not expected. Then I found that was because fetching preview may be called multiple times. Photo Library may provide a low quality image first then provide the full quality image. This issue was solved by read the info dictionary of the fetching preview result. But the black screen issue was still there.

Maybe the AVPlayer or PlayerItem was not ready?

if let video {
    player.replaceCurrentItem(with: video)
    
    Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { timer in
        if player.status == .readyToPlay {
            playerItem = video
            timer.invalidate()
        }
    }
}

The result was the same.

Maybe there was still some time between the player was ready and the player was really playing.

if let video {
    player.replaceCurrentItem(with: video)
    
    Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { timer in
        if player.status == .readyToPlay {
            player.play()
            
            if player.timeControlStatus == .playing {
                playerItem = video
                timer.invalidate()
            }
        }
    }
}

The issue solved.

Conclusion

  1. Do not create another AVPlayer each time. Just create an empty player and replace the replaceCurrentItem.
  2. There was time gap between player ready and player real playing. So you need to play the player first when player was ready and then updated the UI after the player was really playing.
  3. I was using a timer instead of a notification as the former was more simple to use.