肇鑫的技术博客

业精于勤,荒于嬉

A Reply to Apple on SiriKit Issues

I installed the profile of Siri for iOS. And the issue did seem to be solved at first. In fact it didn't. It just passed the above example. If we revised the example a little, this issue was still there.

Here is the new example:

  1. in Xcode, download iOS 13.3 simulator, as SiriKit doesn't work in 13.5 simulator at all.
  2. run against "SiriKit Intent" target with Siri in sample project with iOS 13.3 simulator, say iPhone 8 Plus.
  3. open Shortcuts, create a new shortcut with "SiriKit Issue" app and named it as "Find My Fruit".
  4. Hold home button and say "Find My Fruit".
  5. Siri asks:"What fruit do you want to have?"
  6. answer: Green.
  7. Siri asks:"Just to confirm, you wanted 'Green Apple'?"
  8. answer: Yes.
  9. Siri says:"Here is your Green Apple!"

In a real device, the steps 7 to 9 will be:

  1. Siri asks:"Confirm for name?" // The name here was a placeholder, I think. Siri used the placeholder instead of the parameter's value.
  2. answer: Yes.
  3. Siri says:"OK, done. Here is your Green!"

Profile of Siri for iOS reset current Siri setting and fixed some issue

In fact, I have two projects. One is the project I am working on in Chinese. The other is the sample project in English I sent to you as the issue example. Since both of them are in different languages, I switch the language of Siri a lot.

After receiving the email from you, I switched Siri to English and tested. The issue was still there. Then I installed the profile of Siri. The issue was gone after the system rebooted. However, when I changed Siri to Chinese and tested the Chinese project, the issue still existed.

I then removed the profile and rebooted. Then I installed the profile again with Siri in Chinese as current setting. After rebooting, some parts of the issue of the project in Chinese was gone. Then I switched Siri to English. Again, the issue of the English example was back.

My conclusion: installing the Siri profile reset the current setting of Siri, which fixed some issue. But Siri in other languages still have the issue. Maybe Apple should reset Siri's setting every time the language of Siri is changed.

With above conclusion, we could run the issue of the old example by these addition steps:

  1. Remove Siri profile and reboot.
  2. Switch Siri to a language other than English, say Chinese.
  3. Install Siri profile and reboot.
  4. Switch Siri back to English. Since the profile reset the Chinese Siri. The English Siri still has the issue.
  5. Run the steps of the old example again on the original post.

API regression of SiriKit after iOS 13.3

//
//  INStringResolutionResult.h
//  Intents
//
//  Copyright © 2016-2020 Apple Inc. All rights reserved.
//

@available(iOS 10.0, *)
open class INStringResolutionResult : INIntentResolutionResult {

    
    // This resolution result is for when the app extension wants to tell Siri to proceed, with a given string. The resolvedString can be different than the original string. This allows app extensions to apply business logic constraints to the string.
    // Use +notRequired to continue with a 'nil' value.
    open class func success(with resolvedString: String) -> Self

    
    // This resolution result is to ask Siri to disambiguate between the provided strings.
    open class func disambiguation(with stringsToDisambiguate: [String]) -> Self

    
    // This resolution result is to ask Siri to confirm if this is the string with which the user wants to continue.
    open class func confirmationRequired(with stringToConfirm: String?) -> Self
}

Above is the header file copied from Xcode. You can see that for a single string, a developer could use either open class func disambiguation(with stringsToDisambiguate: [String]) -> Self or open class func confirmationRequired(with stringToConfirm: String?) -> Self. However, in my own test, both of them didn't work with single result on iOS later than iOS 13.3. When the results were more than 1 elements, the api worked. When only one element existed, Siri used the placeholder, which was called "name", instead of the parameter's value.

You can see it from differences between the old example and the new one above. For the old one, there were 3 kinds of apples, so Siri listed them successfully. For the new one, there was only one apple, the Green Apple, Siri asked "Confirm for name?" in a real device on iOS 13.6 beta, on an iOS 13.3 simulator, Siri asked "Just to confirm, you wanted 'Green Apple'?".

Final Conclusions

  1. When switching languages, Siri some times, if not all the times, has an issue of using the default sentences to replace the sentences that a developer provides.
  2. Installing Siri profile and reboot may fixed above issue on Siri of current setting. But if you change the language of Siri, the issue is still there.
  3. There were regressions on APIs of INStringResolutionResult. For single result, both open class func disambiguation(with stringsToDisambiguate: [String]) -> Self and open class func confirmationRequired(with stringToConfirm: String?) -> Self wouldn't work on iOS later than iOS 13.3. Siri used a placeholder called "name" instead of the actual value.

Others

Internationalisation issue with Sirikit Custom Intents & iOS 13.4.1

Weight and Line Height of Font between macOS and iOS

When converting text to image, the converted images were different between macOS and iOS. The main differences are font weight and line height.

Font Weight

Thought the font weight and the fonts are the same, on macOS the font result is always thicker. I don't know why. But in my experience, if you use "HelveticaNeue-Light" for iOS, use "HelveticaNeue-Thin" for macOS.

Line Height

The line height of font is even tricky.

Equation

In Apple's doc, Apple gives below graph. We could draw a simple equation from the graph.

line height = ascent + decent + line gap (leading)

textpg_intro_2x

So I did two tests on both iOS and macOS in Playgound.

// macOS
func getFontInfo(_ name:String) {
    let font = NSFont(name: name, size: 17.0)!
    print(font.ascender) // 13.09033203125
    print(font.descender) // -3.90966796875
    print(font.leading) // 0.0
    print(font.ascender - font.descender + font.leading) // 17.0
    
    let layoutManager = NSLayoutManager()
    print(layoutManager.defaultLineHeight(for: font)) // 20.0
}

getFontInfo("Helvetica")
// iOS
func getFontInfo(_ name:String) {
    let font = UIFont(name: name, size: 17.0)!
    print(font.ascender) // 15.64033203125
    print(font.descender) // -3.90966796875
    print(font.leading) // 0.0
    print(font.ascender - font.descender + font.leading) // 19.55
    print(font.lineHeight) // // 19.55
}

getFontInfo("Helvetica")

From the two tests, we could draw two conclusions:
  1. The equation on iOS was balanced, but on macOS was not.
  2. For the same font with the same weight, the ascender were different.

I didn't know why those happened. So I sent an "Apple Developer Technical Support". Here was the reply from Apple.

apple's reply

According to Apple, if I wanted to use the equation, I should use Core Text framework. But in fact Apple didn't provide line height in Core Text.

Then I did another two tests.

// macOS
func getLineHeightForFontName(_ name:String) {
    let font = CTFontCreateWithName(name as CFString, 17.0, nil)
    
    print(CTFontGetAscent(font)) // 13.09033203125
    print(CTFontGetDescent(font)) // 3.90966796875
    print(CTFontGetLeading(font)) // 0.0
}

getLineHeightForFontName("Helvetica")
// iOS
func getLineHeightForFontName(_ name:String) {
    let font = CTFontCreateWithName(name as CFString, 17.0, nil)
    
    print(CTFontGetAscent(font)) // 13.09033203125
    print(CTFontGetDescent(font)) // 3.90966796875
    print(CTFontGetLeading(font)) // 0.0
}

getLineHeightForFontName("Helvetica")

From all four tests, we could get the conclusions:
  1. Though on iOS, the equation was balanced. The ascent property was modified by Apple.
  2. On macOS, the line height was modified by Apple.
  3. From the above two conclusions, both NSFont and UIFont were not trusted. The only trusted line height was something we get from Core Text.

Line Height

#if os(macOS)
func getLineHeight(_ font:NSFont) -> CGFloat {
    let ctFont = CTFontCreateWithName(font.fontName as CFString, font.pointSize, nil)
    return CTFontGetAscent(ctFont) + CTFontGetDescent(ctFont) + CTFontGetLeading(ctFont)
}
#else
func getLineHeight(_ font:UIFont) -> CGFloat {
    let ctFont = CTFontCreateWithName(font.fontName as CFString, font.pointSize, nil)
    return CTFontGetAscent(ctFont) + CTFontGetDescent(ctFont) + CTFontGetLeading(ctFont)
}
#endif

Others

NSTextView Best Practice

Text Programming Guide for iOS

Cocoa Text Architecture Guide

Core Text - Calculating line heights

NSTextView Best Practice

Initiate Font

Unlike UITextView, you cannot assign font in NSTextView with empty string in Interface Builder. There are many ways to set the font, like to set the font of NSTextView.textStorage.font or set the attributedString of NSTextView.textStorage or just set the attributes. However, those methods are all imperfect.

Working With Input Method

The answers above are all not working with input method. The cursor indicator will still be the unset font's style and the font only applied when you have finished inputing.

The only correct way is to set the typingAttributes property of NSTextView.

inputTextView.typingAttributes = [
    .font : NSFont.userFont(ofSize: 17.0) ?? NSFont.systemFont(ofSize: 17.0),
    .foregroundColor : NSColor.labelColor
]

Don't forget to set the foregroundColor as well, or you textview won't work with light/dark mode switching.

Undo

It is easy to turn on undo in Interface Builder. However, you should know that every time you set string or attributedString property, the undo operations are clear.

You may notice that if the textview is in a modal window, its view controller is presented by presentAsModalWindow(_:), the undo operations will not work as expected.

NSTextDelegate And NSTextStorageDelegate

Both NSTextDelegate and NSTextStorageDelegate work with NSTextView, however there are slightly differences between those functions.

// NSTextDelegate
func textDidBeginEditing(_ notification: Notification)
// only runs once when a user first starts editing.

func textDidChange(_ notification: Notification) 
// only runs when a user edits in the textview.

// NSTextStorageDelegate
textStorage(_ textStorage: NSTextStorage, willProcessEditing editedMask: NSTextStorageEditActions, range editedRange: NSRange, changeInLength delta: Int)
// and
func textStorage(_ textStorage: NSTextStorage, didProcessEditing editedMask: NSTextStorageEditActions, range editedRange: NSRange, changeInLength delta: Int)
// runs every time a user edits, also runs when the developer sets string property in code.

Mixing Fonts Causing the Whole Line Moves

Jun-20-2020 19-50-37

This issue happens when using texts with emoji or texts in different languages. Both TextEdit.app and Pages.app have this issue. Changing text size over 22 may solve this issue. There is no other method to solve this issue yet.

Others

Weight and Line Height of Font between macOS and iOS