Apple has built a new AI infrastructure to deliver AI features introduced in its devices and is utilizing resources available in Google’s cloud infrastructure. Apple’s new AI backend includes:
- A homegrown foundation model.
- Apple Servers.
- Server processors based on its M-series chip designs used in PCs.
Apple has created a Private Cloud Compute infrastructure, which hosts Apple Servers running on Apple’s homegrown chips. The hardware and software offerings are rolled up into a product called Apple Intelligence.
Details about Apple’s machine-learning framework, called AXLearn, on GitHub points to a hybrid AI approach, with Apple combining its homegrown Apple servers with the capabilities of Google Cloud. Google did not respond to requests for comment on that topic.
The AXLearn framework “builds on top of JAX and XLA and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs,” Apple said in a blog entry.
Apple has been working on AI for some time, and this new infrastructure is a significant step forward. The company’s top rivals, Google and Microsoft, have been in the AI game since late 2022.
Apple has two new homegrown AI models, including a 3-billion parameter model for on-device AI and a larger LLM for servers with resources to answer more queries. The ML models – developed with TensorFlow – were trained on Google’s TPUs. TensorFlow was developed by Google and is used extensively across its AI software stack deployed on TPUs.
Apple’s AXLearn AI framework, used to train the homegrown LLMs, creates Docker containers that are authenticated to run on the GCP infrastructure. AXLearn supports the Bastion orchestrator, which is supported only by Google Cloud.
“While the Bastion currently only supports Google Cloud Platform (GCP) jobs, its design is cloud-agnostic, and in theory, it can be extended to run on other cloud providers,” Apple stated on its AXLearn infrastructure page on GitHub.
The new AI hardware marks Apple’s return to server hardware after it discontinued its last dedicated server product, Xserve, which ran on Intel server chips, in 2011. Apple positioned its Mac Pro workstation as a server alternative.
“These models run on servers we have especially created using Apple silicon. These Apple silicon servers offer the privacy and security of your iPhone from the silicon on up, draw on the security properties of the Swift programming language, and run software with transparency built in,” said Craig Federighi, Apple’s senior vice president of Software Engineering, at the company’s WWDC conference.
The company prioritizes privacy in its push to provide AI on its devices. The hosted AI models delete user data after answering queries.
Customers can use those local models to generate images and documents and provide easier document access. Users are authenticated through security mechanisms such as fingerprints or face recognition on PCs or phones, which are also used to authenticate AI server connections.
“Private Cloud Compute allows Apple intelligence to flex and scale its computational capacity and draw on even larger server base models for more complex requests while protecting your privacy. These models run on servers we have specially created using Apple silicon,” Federighi said.
Apple also announced a tie-up with ChatGPT, which is available to users looking for stronger answers. OpenAI isn’t known for data privacy, so Apple’s devices will include prompts asking user permission to send data to ChatGPT.