How do we use AI?

Artificial Intelligence (AI) refers to a set of technologies that use advanced computer algorithms to solve particular set of tasks commonly associated with human-level intelligence. For example, understanding meaning of text is one such task. The way how this is accomplished by a computer program is referred as model. There are many commercial and open-source models, each solving specific task and have different properties making it useful for one context or another. For example, some models work best for Chinese text, while others excel at financial time series prediction, while others can recognize text on images.

In our products, we are using numerous models throughout the whole system, starting from your device (e.g. iPhone) and all the way to the cloud backend systems that store your data. Here are some of the tasks we are solving with AI models:

• Parsing text from image
• Image understanding
• Extracting price and product information from image and text
• Fraud detection
• Barcode scanning
• Search

To accomplish these tasks we are using in-house, open source as well as proprietary technologies, software programs and artifacts (e.g. model weights). Some of them run on your device (e.g. iPhone) and your data does not even leave your device. For example, barcode scanning is done fully on device. However, some of the most advanced models require significant computation power and run in our private cloud. Your data never leaves our systems. Learn more in our Privacy Policy.

Some of the technologies in this category include:

• Vision by Apple
• Natural Language by Apple
• SensitiveContentAnalysis by Apple
• PhotoDNA by Microsoft
• DeepSeek
• Llama by Meta
• Qwen by Alibaba
• Gemma by Google
• LLaVA
• OpenCV
• PyTorch
• llama.cpp
• mistral.rs
• XGBoost

More specifically, we are solving these tasks:

• Image segmentation
• Image and Text deduplication
• Image and Text search
• Image and Text semantic content understanding
• Image and Text entity extraction
• Image OCR (Optical Character Recognition)

The specifics on how each of these models work is beyond the scope of this article. We would only mention here that these technologies rely heavily on linear algebra and highly parallel computation. These methods often draw inspiration from the way how real neural networks accomplish the same tasks. If you would like to learn more, we advise you to look into following material:

• “Deep Learning”, Ian Goodfellow, Yoshua Bengio, Aaron Courville
• “Computer Vision: Algorithms and Applications”, Richard Szeliski
• “Speech and Language Processing” Dan Jurafsky, James H. Martin
• “Principles of Neural Science” Eric R. Kandel, John D. Koester, Sarah H. Mack, Steven A. Siegelbaum

Field of AI is rapidly advancing and we are working hard to bring AI solutions and to contribute back to open source and research communities. It is our passion to deliver the best service to you using these technologies, while maintaining Safe, Secure and Private service. We strive to create a truly delightful experience.