肇鑫的技术博客

业精于勤,荒于嬉

Something Apple Didn't Say Right On Photos Framework

I am developing an app basing on Photos framework. However, I find that Photos framework is something mainly for Photos.app and not well documented.

I will listed them for further references.

System Default Alert Message May Confuse Users.

When removing an album from photo library, you must use class func deleteAssetCollections(_ assetCollections: NSFastEnumeration). The system will automatically represents an alert to ask the user whether to remove the album. However, there is a message says that only the album is removed, the photos inside the album are kept.

The message is accurate for Photos.app. However, if we want to remove the photos together with the album, there is no API for that, so we have to call the remove API twice, one for photos, the other for album. There will be two alerts shown and the alert message mentioned may confuse the user as the photos are removed this time.

Folders

Folder can contain other Folders, but not its parents.

Folder, which is called PHCollectionList is said can contain other folders. That leads me to say what if folder A contains folder B and B contains A, which make the relation recursive. So I created an example, and I found that when A contains B and B contains A, photo library would rise an error. So the recursive relation is not allowed.

The Collection operations are all movement, not copying.

When putting other collections into one folder, whether you use addChildCollections(_:) or insertChildCollections(_:at:) is irrelevant. The collections are just moved inside the folder, not copied.

This rule also makes there is not duplicated folders in photo library.

The result of fetchCollections(in:options:) is shallow.

Although fetchCollections(in:options:) says it returns "By default, the returned PHFetchResult object contains all collections in the specified collection list." The document is not accurate. It only returns the shallow collections, not all collections. For example, we have three folders, A, B and C. The relations of them is: A -> B -> C, A contains B, B contains C.

let collections = PHCollection.fetchCollections(in: A, options: nil)
print(collections.count)
// 1. Print 1 instead of 2. As folder B is the only collection with shallow search.

Other constrains of folder operations

PHPhotoLibrary.shared().performChanges {
    let requestA = PHCollectionListChangeRequest.creationRequestForCollectionList(withTitle: "FolderA")
    let requestB = PHCollectionListChangeRequest.creationRequestForCollectionList(withTitle: "FolderB")
    let folderB = requestB.placeholderForCreatedCollectionList
    let requestC = PHCollectionListChangeRequest.creationRequestForCollectionList(withTitle: "FolderC")
    let folderC = requestC.placeholderForCreatedCollectionList
    requestA.addChildCollections([folderB] as NSFastEnumeration) // move B to A, works
    requestB.addChildCollections([folderC] as NSFastEnumeration) // move C to B, works
    requestA.addChildCollections([folderC] as NSFastEnumeration) // move C to A, doesn't work
} completionHandler: { success, error in
    if success {
        print("success")
    } else {
        print(error?.localizedDescription ?? "nil")
    }
}

Result in Photos.app: A -> B -> C
test_1

Line 9 doesn't work. I guess because C has been already in A by contained by B.

However, code below works.

PHPhotoLibrary.shared().performChanges {
    let requestA = PHCollectionListChangeRequest.creationRequestForCollectionList(withTitle: "FolderA")
    let requestB = PHCollectionListChangeRequest.creationRequestForCollectionList(withTitle: "FolderB")
    let folderB = requestB.placeholderForCreatedCollectionList
    let requestC = PHCollectionListChangeRequest.creationRequestForCollectionList(withTitle: "FolderC")
    let folderC = requestC.placeholderForCreatedCollectionList
    requestA.addChildCollections([folderB] as NSFastEnumeration) // move B to A, works
    requestA.addChildCollections([folderC] as NSFastEnumeration) // move C to A, works
    requestB.addChildCollections([folderC] as NSFastEnumeration) // move C to B, works
} completionHandler: { success, error in
    if success {
        print("success")
    } else {
        print(error?.localizedDescription ?? "nil")
    }
}

Apple avoids structural complexity by design.

In Photos.app on macOS, you can only move the collections besides the target folder on the same level. This avoid the complexity of the model structure. In fact, you can move any folder to the target folder. However, that will make not every move operation working, which may confuse user that doesn't familiar with the rules.

完美解决Xcode在预览iOS应用SwiftUI View时,占用内存过多的问题

我的Mac配置是M1,16GB内存,512GB固态硬盘。原以为足够使用了,顶多是偶尔内存不足,用磁盘虚拟一些就足够了。可没成想,这两天调试iOS应用,动不动内存压力就黄了。

虽然我没有感知到系统变慢,但是看着内存压力变黄就是不爽。于是想追究一下具体的原因。结果让我大吃一惊!

为什么Xcode在iOS应用的预览时,会占用额外的内存?

所谓占用额外的内存,是相比较同样的代码,macOS版本应用预览时占用的内存。

macOS应用在预览时,因为Xcode本身就是在macOS运行的,所以无需模拟系统,只需要运行程序就可以了。所以相比于模拟iOS应用,少了模拟了iOS系统,节省了大量的内存。

根据上面的原因,我首先想到的是可以使用模拟较低版本的iOS系统。因为众所周知,版本越新的系统,可能消耗的资源就越大。目前Xcode自带的模拟器是iOS 16.4,由于我的新应用最低支持到iOS 16,所以我选择额外下载iOS 16.0的模拟器。

经过测试我发现,使用iOS 16.0模拟器,相对于iOS 16.4的模拟器,可以节省约5%的内存。16GB*5%=0.8GB。

此外,我之前默认都是使用iPhone SE 3进行模拟,因为这个也是我目前在使用的手机。不过iPhone SE 3本身是4GB内存,所以模拟也需要4GB内存,而iOS 16支持的手机中,内存最小的是iPhone 8,于是我将iPhone SE 3换成iPhone 8,结果发现内存的占用又降低了一些。

将模拟器从iPhone SE 3换成iPhone 8之后,可以再多节省约2%的内存。两次节省合计16GB*(2%+5%)=1.12GB,节省内存超过1GB。

大功告成了吗?

原来只需要换掉模拟器,就可以节省超过1GB的内存。事情真的只有这么简单吗?并没有!

平常我们在调试应用时,都是用Xcode写好界面和功能,在预览里看着差不多了,然后上真机上跑的。

xcode_preview

如图,我是在图中2所示的那样,通过制定模拟器型号来进行模拟的。但是我发现,最终占用的内存大小,要比我预期的大。

进一步分析,我发现,原来图中1部分的内容,会对于内存占用产生巨大的影响。

Xcode在预览时存在bug,导致模拟器内存占用翻倍

我们先做一番测试,然后你就知道问题出在哪里了。

  1. 首先我们维持2不变,然后将1的内容也改成iPhone 8。然后退出Xcode,重新开。(退出Xcode并重新开的目的是恢复真机模拟所占用的缓存)
  2. 点击预览,当预览呈现时,我们查看内存占用。此时内存的占用符合我们的预期。
  3. 将1的内容改成真机iPhone SE 3。因为我们模拟是指定的模拟器,此时3的内容应该不变。
    1. 但实际上,你会发现3位置的标签在转圈,这意味着模拟器在刷新。而这本来是不应该发生的。
    2. 转圈结束后,我们再次查看内存,发现内存占用大大增加。

这说明,Xcode出了bug,在我们指定模拟器的情况下,1的内容是否改变,应该不影响2、3的模拟器的,但实际上,虽然3表面上还是2的模拟器的最终结果,但是Xcode还额外开了1的模拟器,这导致模拟器打开的数量翻倍,内存占用大大增加。

验证上面的提到的bug

我们可以通过如下的步骤验证上面提到的bug:

  1. 将2中指定模拟器的代码注释掉。这样3中的模拟器,就会伴随1的变化而变化。
  2. 关闭Xcode,并重开,然后选择预览。
  3. 将1中的iPhone SE 3,换成iPhone 8,然后重新回到步骤2。

我们发现,无论是从1到2,还是从3到2,最终占用的内存都大大低于上面的情况。换句话说,也就是当2不设置时,因为1与3是相同的,每次都会只打开1个模拟器,因此内存占用的区别仅仅是iOS版本和iPhone版本区别,而不涉及到重复打开模拟器的问题。

不完美的解决方案

那么,我们为了保证同时还要调节真机的需要,是不是就只能妥协。不指定模拟器的型号,而选择与真机一致这种次优的解决方案了呢?

完美解决方案

并不是!我最终找到了完美的解决方案。

我们知道,Xcode之所以能调用多个模拟器,是因为系统中存在多个模拟器。那么如果我们通过手段,删除掉多余的模拟器,那是不是Xcode就不会调用了呢?抱着试试看的想法,我删除了所有模拟器,仅保留iPhone 8 iOS 16.0这一个。然后再次进行测试。

结果符合预期!因为系统中不再包含iPhone SE 3的模拟器,所以即便1切换为iPhone SE 3,3中的标签也不会有任何变化。因为iPhone 8是唯一的模拟器,所以任何时候,都只会用iPhone 8进行模拟。2中的指定模拟器也是没有必要的了。

至此,Xcode在模拟iOS应用的SwiftUI View预览时,占用内存过高的问题完美解决了。

实际使用中,我的内存剩余,从不足40%,恢复到接近70%。压力也不再变黄了。
测试时使用内存压力的手段是在终端中输入memory_pressure命令,查看最后一行的剩余内存。

其它节省内存的技巧

我发现Xcode有一个问题,正常退出Xcode之后,有时SourceKitService并没有伴随退出,而是继续留驻在内存中。它会占据1GB左右的空间。解决办法是在任务管理中,手动终止这个进程。

为了解决这个问题,我手动写了一个工具。会在在Xcode关闭之后,查看是否有SourceKitService遗留,有则自动结束,减少内存占用。如果你也需要,可以尝试使用。

SwiftUI view recreated may cause bugs what are hard to debug

Today, I encountered an issue that VideoPlayer turned to blank when iPhone screen was rotated.

The VideoPlayer was in a modal view that created with fullScreenCover. The sample code was like this:

import SwiftUI
import AVKit

struct TopView: View {
    private let player = AVQueuePlayer(playerItem: nil)
    
    var body: some View {
        VideoPlayer(player: player)
            .onAppear {
                if let videoURL = Bundle.main.url(forResource: "sample", withExtension: "mov") {
                    let playerItem = AVPlayerItem(url: videoURL)
                    player.replaceCurrentItem(with: playerItem)
                    
                    Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { timer in
                        if player.status == .readyToPlay {
                            player.play()
                            
                            if player.timeControlStatus == .playing {
                                timer.invalidate()
                            }
                        }
                    }
                }
            }
    }
}

Using Publisher for UIDevice.orientationDidChangeNotification

At first, I thought I should recreate the player when the screen was rotated.

import SwiftUI
import AVKit

struct TopView: View {
    @State private var playerItem:AVPlayerItem?
    
    private let player = AVQueuePlayer(playerItem: nil)
    private let publisher = NotificationCenter.default.publisher(for: UIDevice.orientationDidChangeNotification)
    
    var body: some View {
        if playerItem != nil {
            VideoPlayer(player: player)
                .onReceive(publisher, perform: { _ in
                    self.player.pause()
                    self.playerItem = nil
                })
        } else {
            ProgressView()
                .onAppear(perform: setPlayer)
        }
    }
    
    private func setPlayer() {
        if let videoURL = Bundle.main.url(forResource: "sample", withExtension: "mov") {
            self.playerItem = AVPlayerItem(url: videoURL)
            player.replaceCurrentItem(with: playerItem)
            
            Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { timer in
                if player.status == .readyToPlay {
                    player.play()
                    
                    if player.timeControlStatus == .playing {
                        timer.invalidate()
                    }
                }
            }
        }
    }
}

However, the new code didn't work. I added more debug point and finally found that TopView was recreated when the screen was rotated. Since the view was recreated, the player in onReceive was newly created, it couldn't stop the playing of the previously played item.

The issue was because, unlike other structs and objects, which could automatically released when container view was released. AVPlayer hold its owned reference when playing. This behavior caused the SwiftUI view was not released properly.

Use Binding from parent view

The solution was easy. Since I wanted the player to be constant, I should set it up in the parent view.

import SwiftUI
import AVKit

struct TopView: View {
    @Binding var player:AVQueuePlayer
    
    var body: some View {
        VideoPlayer(player: player)
            .onAppear {
                if let videoURL = Bundle.main.url(forResource: "sample", withExtension: "mov") {
                    let playerItem = AVPlayerItem(url: videoURL)
                    player.replaceCurrentItem(with: playerItem)
                    
                    Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { timer in
                        if player.status == .readyToPlay {
                            player.play()
                            
                            if player.timeControlStatus == .playing {
                                timer.invalidate()
                            }
                        }
                    }
                }
            }
    }
}

Now everything worked fine.

Final Thoughts

Some objects hold their own references as strong. Those objects may keep view from release and cause bugs. We should using Binding to create those objects in a higher view which is not recreated. Then the bugs are fixed.