The Apple Watch Ultra’s Action Button is powered by some awesome new App Intents APIs. You will be able to build your own apps to integrate with it, like a hockey app that uses the button to record goals! And for users, the button can kick off any Shortcut you want!
Mind blown: You can turn on spell checking in Xcode. Edit -> Format -> Spelling and Grammar. This works on comments and identifiers, and is quite aware of capitalization conventions, etc.
So. Many. Typos. Found.
macOS devs and AX users: Check out the macOS 10.15.4 beta for head pointer: A new feature that lets you control your macOS mouse pointer with your head. My team and I have been working on this for a while, and we are excited for you to try it! Feedback welcome.
My team, macOS Accessibility, is hiring! We work on extremely impactful AX products, including VoiceOver, Switch Control, and Head Tracking. We are also building the next generation of AX tools and frameworks with SwiftUI. Interesting? Drop me a line.
SwiftUI is a big deal for Accessibility. Because of the high level abstractions SwiftUI uses, we’re able to do more for you automatically. And it opens up some awesome possibilities going forward, as well.
One small but critical change in Monterey: Switch Control, the Accessibility Keyboard, and other Motor Accessibility features now work in special boot modes. On Intel, this includes Recovery mode. On Apple Silicon, it includes the FileVault login screen!
I worked on the Accessibility side of the new SwiftUI framework. Happy to chat about this during WWDC. We are also giving a talk about Accessibility and SwiftUI (10 AM on Friday in Hall 4) and a lab immediately afterwards. Come and learn!
🧵 This is a massive year for Accessibility in SwiftUI. Here is an overview for you. Much of it is in our two talks, which you should totally watch for more information. There were so many delicious new APIs, though, that not everything made it into the talks! Here goes ...
Questions about SwiftUI Accessibility? Come meet myself and team at the WWDC labs. We are here to help you, and also to learn from your experience building Accessible SwiftUI apps. You can sign up in the Developer app.
A personal update: I’ve taken a new and exciting job. I’ll be managing a small team of crazy talented folks within Apple’s Proactive intelligence group. We are working on Shortcuts and other features making your Apple devices more powerful and productive than ever.
We put in quite a bit of effort to make sure that existing plugins work in Apple Silicon Macs. For example, Automator loads existing x86_64 third party actions using a separate process for their execution and UI over XPC. “It just works.” But “just” is doing a lot of work there.
Hi folks, my team at Apple is hiring. macOS and iOS Accessibility. That means VoiceOver, Switch Control, Accessibility Keyboard, and lots more. Ping me if you are interested. Unfortunately, no remote at the moment.
My team and a bunch of other folks have been working on App Intents for a long time. It’s a really slick API and it makes “programmable software” incredibly fun and easy to build. I hope you love it. Check out my session tomorrow: “Dive into App Intents.”
Exciting times in App Intents and Shortcuts land! We're ready to hire an experienced engineer to join our team and work with us building the next generation of apple intelligence and programmable software.
Hi folks: I’m hiring interns on team Shortcuts / Proactive, for this summer. Come enable people to do amazing things with their Apple devices. Build some bicycles for the mind with us!
You’d need to come to Cupertino, CA for the summer, and already be eligible to work in the US.
My team is starting early discussions for internships for next year. If you are a student interested in working on App Intents, Apple Intelligence, and the Shortcuts infrastructure that powers it, drop me a DM please! Note that we cannot help with immigration. Local to Cupertino.
Feedback is critical for us to get new ideas and frameworks right. This is particularly important for relatively young ones like SwiftUI.
If you have issues/requests re: Accessibility in SwiftUI, please, please file a feedback. Feel free to drop me a line with the number.
We’ve also got a new Accessibility API for SwiftUI apps, which we’ll discuss in the session. For the first time, it’s a unified accessibility API across all platforms.
App Intents update thread!
1. With Beta 3 you’ll have access to the new “some IntentResult” syntax for returning results from your Intents, along with the OpensIntent, ReturnsValue, and ProvidesDialog protocols. Helps us extract return type at build time without running your app
Team and I have been working on this feature for a long time. We are so proud of what it can do in the hands of
@TheSwitchMaster
, and other amazing folks! Still feels like science function sometimes. This kind of thing used to require crazy expensive hardware.
Head Pointer is a new macOS accessibility feature. Here’s how to set it up! …
SPOILER: You don’t need to buy any special hardware or put stickers on your face. :)
#GAAD
#Apple
Exactly. In fact, myself and my team, and other teams at Apple too, work _super hard_ to make it incredibly easy for you to make your apps accessible. We spend all day (and some nights!) thinking about this specific bar, and how to lower it as much as possible.
"The biggest misconception that people have about making apps accessible is thinking that it’s a lot of work and it’s really difficult, but it’s not."
#WWDC21
#Accessibility
This means folks with motor disabilities will no longer have to chose between disabling FileVault, or sharing their password with another party, just to unlock their Mac on boot!
This is a huge improvement in security and privacy for disabled folks.
How is your adoption of App Intents, App Shortcuts, and Focus Filters going? If you’re running into outstanding issues with the latest betas, or limitations blocking you from building the Awesome Thing, drop me a line and let me know. Loving what I’m seeing already on Twitter ❤️
I can’t wait to finally share everything we have been working on with care and passion for the last year. I hope you’ll love all the features and improvements to SwiftUI.
I’ll be hanging out in the Digital Lounge and Labs ready to answer all your questions.
Happy
#WWDC21
week!
It was so great to introduce App Intents and Focus Filters this week, and to see community tutorials and adoptions getting started early! If you have any questions and requests, do file feedback if you can, but also feel free to drop me a line directly.
Student peeps, we are officially looking for an intern for the summer of 2023. Come work on Shortcuts and App Intents with us, and help empower an incredible community of people. You’d have to come to Apple Park, and unfortunately we can’t help with immigration. My DMs are open!
This is a really good point. In fact I think a big part of a senior developer’s skill is managing their own emotions. Speaking for myself, at least, there are so many flavors …
we think about debugging as a technical skill (and it absolutely is!!) but a huge amount of it is managing your feelings so you don't get discouraged and being self-aware so you can recognize your incorrect assumptions
I’m incredibly proud to be a tiny part of this project, and honored to work with these folks. Emotionally, it is also incredibly exciting to see and participate in a sea-change in computing like this. Feeling like a very lucky nerd today.
To use this, make an Intent that conforms to the StarWorkoutIntent protocol:
These APIs are meant for Workout apps, so to put this new button in a mode where it’s controlling your app, you’ll need to at least start with a StartWorkoutIntent.
We’ve got a new open position for engineering SwiftUI Accessibility across all platforms. This is an opportunity to shape the way accessible apps are are built for the next 10/20 years. If you are interested, drop me a line! Must be able to move to Cupertino, CA later next year.
This is a result of work by myself and team, as well as a lot of very smart folks working at lower layers. It represents many years of deep technical investment in Apple Silicon and other core technologies coming together beautifully to create a big win for Accessibility.
Also, there are a bunch of App Intents related bug fixes in iOS 16.1 Beta 1. If your app adopts App Intents, please try it out and let me know if your issues are resolved, or if you encounter any new ones!
In all seriousness, thank you to those that have tried AppIntents and built some incredibly cool things with it. And an extra cherry on top to those that have battled the teething problems, filed FBs & samples, complained on Twitter. It’s above and beyond & I really appreciate it
So it’s uh … a big day for App Intents. If you are around Apple Park for today, come by the Machine Learning and AI area if you want to say hi! I love talking with our adopters. It makes my year.
@twostraws
@ViewBuilder
The
@ViewBuilder
thing is a general Swift feature. Protocols that use function builders now have the function builder propagate down to implementations of the protocol’s requirements automatically. This applies to the new builders added to SwiftUI this year as well.
Want to come talk about App Intents? Come chat with myself and my team at Ask Apple. We can answer your questions, and would love your feedback too.
I’ll be around for the Apple Watch Q&A on Monday 2 PM PST, and the App Intents Q&A on Wednesday 2 PM PST
Super proud of what my colleagues have pulled off with the new Apple Silicon Macs. Congrats folks, I am so lucky to work with all of you, and be a part of projects like this.
I will miss working directly on Accessibility. It was and is incredibly meaningful and satisfying work that has totally changed the way I see software and its impact on human lives. It’ll forever shape what I build, and how I build it. I’ll always be looking for intersections.
Check out my team's talk for this year:
We introduce a TON of new features in App Intents. One huge one is the ability to build user-configurable Widgets with App Intents instead of SiriKit! No more definition files, just pure, beautiful code.
Once you’re done with that, you can create an AppIntent result with an actionButtonIntent to set the next thing the button will do, to any other intent. Or, if you’re changing the button’s behavior based on something that happened in your app’s UI, you can use donations to do so.
Very cool! When you do a install of Big Sur you get this screen during on boarding. When you click one of the categories you are shown features that can be enabled and demo of how that feature can help. Very cool!!
#WWDC20
#BigSur
iOS 14 has a new Back Tap feature in Accessibility and it's wild. You can perform quick actions by double- or triple-tapping the *back of your iPhone*. Literally quick taps on the back; works with a case on.
These include system actions as well as custom shortcuts.
On a related note, having a community like you folks around us, taking these tools and turning them into awesome software, is such a privilege. Thank you all for your work, the mind bicycles you build, and the humans you empower with them.
Finally back from a much needed post-WWDC break and now catching up on all your mentions and notes. It was great to meet so many fellow nerds in person at Apple Park!
On iOS 15 SwiftUI will automatically adjust move and slide transitions for "Prefer Cross-Fade Transitions" accessibility setting. No additional work required for developers! (Attached video is showing it in action in the simulator)
The new Home Screen customization trend is a really good thing and makes me really happy. Not feeling many of the aesthetics, but that doesn’t matter. Folks should be able to express themselves through the things they use every day.
1. There is now a new Inspector for GUI editing of Accessibility info in SwiftUI. Even better, there is a new runtime tab which lets you see SwiftUI Accessibility information in your Previews, in real time. This'll make it an order of magnitude quicker and easier to get it right!
Thank you, third party devs, for all the iOS 13 adoption, including Dark Mode and Voice Control. I’m blown away, and so proud to be a small part of enabling what you do. We have a community of developers like no other.
Too many 3rd party apps sleep on this pan gesture to bulk select, which is a shame because it’s a great way to quickly select stuff.
Super easy too:
func collectionView(_ collectionView: UICollectionView,
shouldBeginMultipleSelectionInteractionAt indexPath: IndexPath) -> Bool
I feel lucky and incredibly excited to get to build on the “bicycle for the mind” concept that I’ve been hooked on since I worked on Automator in 2007. Your devices are there to enable you to do incredible things. It’s an old idea, but it is alive and thriving on this team.
A huge thank you to the folks on my team that made all of this possible, including the brilliant
@nathantannar4
who did our talk this year, and many of the other engineers on on the SwiftUI and Accessibility teams. So lucky to work with some really incredible people.
@kzzzf
AnyView also has performance implications. Instead, consider using if statements inside of view builders, or use ConditionalContent directly. That allows the switch but preserves static typing.
2. There is a new `accessibilityRepresentation` API that will help you make custom controls accessible. This lets you tell accessibility to treat some arbitrary custom UI as a standard control, giving you the benefit of all our work for that control.
One more time, for the evening folks. Another important note: we are working hard to build a diverse and inclusive team. If anyone has connects to under-represented communities, I would really appreciate it if you passed this along.
Hi folks: I’m hiring interns on team Shortcuts / Proactive, for this summer. Come enable people to do amazing things with their Apple devices. Build some bicycles for the mind with us!
You’d need to come to Cupertino, CA for the summer, and already be eligible to work in the US.
We have some awesome videos coming this week. The first two are out now. Check out Dan’s awesome talk for App Intents + Siri integration:
And
@nebelch
’s awesome overview of the App Intents ecosystem:
One of the best parts about working on APIs is that what you folks do with them far exceeds what I could have imagined. That’s why we make ‘em!
Looking forward to getting a chance to meet our lovely adopters.
For those that have adopted App Intents in their apps, or are considering it: What are the main frustrations / limitations you have hit? FB numbers are great, but in this case not required!
@Sommer
Another one: A UI that doesn’t force users to do things in a limited period of time (think an expiring login or form) is an Accessible UI. It’s also just less stressful for everyone!
This is going to be great. Super excited for how much this will help speed up developers doing accessibility work.
Try it out and give us feedback, please!
As always: Please try all this out and file Feedbacks. They really are important. They go directly to me, and I read every one. We need your help to make sure this all works as well as possible, For you, for us, and most importantly for folks that need our Accessibility features.
@marcoarment
If you are willing to keep around the SiriKit Intent code, the CustomIntentMigratedAppIntent protocol will allow you to migrate iOS 16 clients to AppIntents, and continue to support iOS 15 clients via SiriKit intents, without breaking saved shortcuts.
AppIntents devs warning: When running in macOS 13 beta, AppIntents code may fail to build with Xcode 14 RC
Workaround: Build AppIntents code with Xcode 14 beta 6, or on a Mac running macOS Monterey 12 with Xcode 14 RC.
(Release note coming soon)
I also get to work with the incredibly talented
@AriX
, which means I will learn _a ton_. It’s a big change for me, but I can already tell it is going to be a wild and wonderful ride.
Want to talk SwiftUI Accessibility? The SwiftUI Accessibility Watch Party in the Accessibility Lounge, WWDC 21 Slack, starts _now_!
Come say hi, learn, and give us feedback.
For those newly remote, here are a couple things I’ve found helpful over the last couple years of remote working.
1. Keep a specific office / dedicated work space. Train your brain and others that when you are there, you are at work.
2. Communicate CONSTANTLY. Over communicate.
Oh, don’t forget the terrifying anxiety when they are deep in a refactor and you have no idea whether you are riding a good idea and things are about to click, or whether you are following a bad idea right off a cliff. Love that one.
Want to talk App Intents? Come watch the session and meet me and the team, at 2 pm Pacific today on Slack! “Meet the Presenter: Dive into App Intents”.
If you want the system to predict/suggest your App Intents, you can implement the PredictableIntent protocol and set up a prediction configuration, specifying combinations of parameters that can be predicted, and how to describe them!
3. There is a new `accessibilityChildren` API to let you create invisible child elements just for accessibility. Great for making `Canvas` or `Image`s accessible, for example! This is similar to manually creating `UIAccessibilityElement` objects, and can be a powerful tool.