Posted on Leave a comment

Arm Server Update, Spring/Summer 2021

As usual, we are overdue for an update on all things Arm Servers! Today’s announcement of the Arm v9 specification is a great time to review the state of Arm Servers, and what has changed since our last update.

First, let’s review our last update. Marvell canceled the ThunderX3 product, Ampere had announced the Altra but it wasn’t shipping, AWS Graviton was available, and Nuvia was designing a processor.

Fast forward to today, and the Ampere Altra’s are now becoming available, with limited stock via the Works on Arm program at Equinix Metal, and some designs shown off by Avantek, a channel supplier. Mt. Snow and Mt. Jade, as they are known, are also formally designated as “ServerReady” parts, passing standards compliance tests.

Nuvia, the startup that was designing a new Arm Server SoC from the ground up, was purchased by Qualcomm, in an apparent re-entry into the Arm Server market (or for use in Windows on Arm laptops?). Don’t forget, they previously had an Arm Server part, the Centriq, though they scrapped it a few years ago. So, it now remains to be seen if Nuvia will launch a server-grade SoC, or pivot to a smaller target-device.

The other emerging trend to cover is the role of Arm in the Edge Server ecosystem, where the trend of pushing small servers out of the datacenter and closer to customers and users is rapidly gaining momentum. In this scenario, non-traditional, smaller devices take on the role of a server, and the energy efficiency, small form-factor, and varied capabilities of Arm-powered single board computers are taking on workloads previously handled by typical 1U and 2U rackmount servers in a datacenter. But, small devices like the Nvidia Jetson AGX, RaspberryPi Compute Module 4, and NXP Freeway boxes are able to perform Edge AI, data caching, or local workloads, and only send what is necessary up to the cloud. This trend has been accelerating over the past 12 – 18 months, so, we may see some more niche devices or SoC’s start to fill this market.

Posted on Leave a comment

Arm Server Update, Fall 2020

The announcement yesterday of the cancelation of Marvel’s ThunderX3 Arm Server processor was a reminder that we were overdue for an Arm Server update!  So, continuing on in our regular series, here is the latest news in the Arm Server ecosystem.

As mentioned, unfortunately it appears at this time that Marvell has canceled the ThunderX3 Arm Server processor that was shown earlier this year, and would have been the successor to the ThunderX and ThunderX2 parts released previously.  The current rumors indicate that perhaps some specialized version of the SoC may survive and be used for an exclusive contract with a hyperscaler, but that means “regular” customers will not be able to acquire the part.  And with no general purpose, general availability part, the ThunderX3 will effectively be unavailable. 

That leaves AWS providing the Graviton processor in the EC2 cloud server option, or Ampere with their current generation eMag Arm Server, and forthcoming Ampere Altra SoC as the only server-class Arm processors left (for now).  The Ampere Altra is brand new, and available from our friends at Packet in an Early Access Program, but no specific General Availability date has been mentioned quite yet.  This processor offers 80-cores or 128-cores, and is based on Arm Neoverse N1 cores. 

There is another processor on the horizon though from Nuvia, a startup formed late last year who is designing an Arm-based server class SoC.  Nuvia has said it will take several years to bring their processor to market, which is a typical timeframe for an all-new custom processor design.  So in the meantime, only Amazon and Ampere are left in the market.

The NXP desktop-class LS2160 as found in the SolidRun Honeycomb could also be considered for some workloads, but it is a 16-core part based on A72 cores.

There is one other Arm Server that exists, but unfortunately it’s not able to be acquired outside of China:  the Huawei TaiShan 2280 based on the HiSilicon Kunpeng 920.  This is a datacenter part that is likely used by the large cloud providers in China, but seems difficult (or impossible) to obtain otherwise.  It is a dual processor server, with 64-cores in each processor, thus totaling 128 cores per server.

As usual, the Arm Server ecosystem moves quickly, and we look forward to seeing what’s new and exciting in our next update!

 

Posted on 3 Comments

Where to Buy an Arm Server

Being Arm enthusiasts and deeply embedded in the Arm Server ecosystem, one of the questions we get asked often is,

“Where can I buy an Arm Server?”

In the past, it was difficult to actually find Arm Server hardware available to individual end-users. Not long ago, the only way to gain access to Arm Servers was to have NDA’s with major OEM’s or having the right connections to get engineering-sample hardware. However, over the course of the past 2 to 3 years, more providers have entered the market and hardware is now readily available to end users and customers. Here are some of the easiest ways to buy an Arm Server, although this list is not exhaustive. These servers all have great performance and are well supported thanks to standards compliance and UEFI.

First up is the Marvell ThunderX, and newer ThunderX2. These chips are sold in servers from several vendors, which come in various shapes and sizes. Some of the examples we’ve found include the Avantek R-series in both 1U and 2U sizes, and the Gigabyte Arm offering that closely match Avantek’s specs. There are High Density designs, single processor and dual processor options, and 10 GBE as well as SFP options available.  ThunderX2’s have been more popular in HPC environments, but even a first-generation ThunderX is a great choice, and still a very powerful machine.  They can be purchased with up to 48-cores, or in dual-processor configurations then containing up to 96 cores.

Another option is the Ampere eMag Arm Server from a company that formed a few  years ago, Ampere Computing.  They ship a turnkey Arm Server that is sold by Lenovo, the HR330A or the HR350A.  Their first-generation platform has 32 Arm cores running at 3.0ghz, 42 lanes of PCIe bandwidth, and 1 TB of memory capacity, and their second-generation product, the Ampere Altra, has up to 80 Arm Neoverse N1 cores.  Current models are available for purchase from their website, or through Lenovo.

Finally, although it is marketed as a workstation, the Solid Run Honeycomb LX2 motherboard can quite easily be repurposed as a proper server.  With 16x A72 cores, support for 64gb of RAM, up to 40gb Ethernet, and PCIe expansion, it can definitely handle medium sized workloads.  It is standards-compliant, making it easy to install your OS of choice, and affordable, thus its a great option for getting started on Arm.

And of course, if buying physical servers and hosting them yourself, or placing them in a datacenter, is not feasible or cost effective in your situation, then our hosted Arm servers are a great option as well!  Our miniNodes Arm servers are certainly more modest in comparison to those mentioned above, but, they are a great way to get started with Arm development, testing existing code for compatibility, or lighter workloads that don’t require quite so much compute capability.

Be sure to check back often for all things Arm Server related!

Posted on Leave a comment

Recap: Building an Arm-Powered IoT, Edge, and Cloud Infrastructure

Intro

At Arm’s annual TechCon event in San Jose, Arm CEO Simon Segars presented a vision of the future where a trillion connected devices interact seamlessly with each other and pass data between the Cloud, the Edge, and the Internet of Things, at a scale unimaginable even just a few years ago. Self driving cars will generate massive amounts of sensor information and data, 5G wireless will enable increased connection speeds and reduced latency, and artificial intelligence will provide scientific breakthroughs in materials, technologies, medicines, and energy. This vision of the future state of the connected world is something we have heard about for several years now, with countless written articles, interviews, social media posts, conference talks, and various other forms of media addressing the topic.

However, when seeking out real-world examples of this architecture in practice to help learn and understand how the bits and pieces work together, we came up empty. There were no purpose-built sample projects, pre-written code examples, or other working prototypes of these principles available. Surely there are some internal, private teams building out this type of infrastructure for specific use-cases and organizational needs, but there were no public / open projects to learn from.

Thus, it was time to take action, and build a prototype infrastructure ourselves! With the help of the Arm Innovator Program, we set out on a journey to develop a proof-of-concept that encapsulates as many of these concepts as possible, leveraging currently available technologies and showcasing Arm’s diverse portfolio of products and ecosystems. With help from the Works on Arm program via Packet.com, we began brainstorming.  Our goal was to deploy IoT endpoints to a handful of locations around the world, and capture environmental data via sensors on those devices. From there, we wanted to feed that data to a local Edge Server, which would be responsible for translating the data to a usable format and sending it further upstream, to a Cloud Server functioning as a data warehouse and visualization platform.

In this article we’ll take an in-depth look at the project, and detail the key technologies to give a better idea of what this kind of system entails. I’ll also provide a summary of our lessons learned, which hopefully help you to build and iterate faster, and avoid some potential pitfalls along the way.

Design

When thinking about the design of this project, we wanted to keep things simple, as the purpose of this exercise it to demonstrate capability and build a proof-of-concept, but not an actual product shipped to real, paying customers.  Thus, we made hardware and software selections based on cost and availability, as opposed to “most appropriate” for the intended use. We also knew we would have relatively small data-sets, and reliable power and internet connectivity for all of our devices.  Your real-world IoT deployments may not have these luxuries, so, your hardware and software selections may not be as straightforward as ours were.  Many IoT projects have to be tolerant of lost network connectivity, unreliable power delivery, or harsh environmental conditions.  But we were fortunate to have consistent power and internet.  Let’s go through our inventory of Arm-powered hardware and software, keeping in mind the rather ideal conditions we’ve got:

1. IoT Endpoints

Hardware

  • Raspberry Pi 3B+
  • Sparkfun Qwiic HAT
  • Sparkfun Lightning Detector
  • Sparkfun Environmental Combo Sensor
  • Sparkfun GPS Sensor

Software

  • Arm Mbed Linux OS
  • Arm Pelion Device Management

 

2. Edge Nodes

Hardware

  • Linaro / 96Boards HiKey, and HiKey 960

Software

  • Linaro Debian Linux

 

3. Cloud Server

Hardware

  • Ampere eMAG, hosted by Packet.com

Software

  • Debian Linux
  • InfluxDB
  • Grafana

 

As you can see, we have made some selections that fit our small project well, but as mentioned may not be suitable for all IoT use cases depending on your project’s environmental conditions.  However, let’s start detailing the items, beginning with the IoT Endpoint.  We’re using a Raspberry Pi 3B, a Sparkfun Qwiic HAT, and Sparkfun sensors to capture Temperature, Humidity, Barometric Pressure, CO2, and tVOC (volatile organic compounds).  We have lightning detection capability (currently not being used, but, available) as well, and GPS so that we can determine precisely where the Endpoint is located.  As for software, because these devices are out in the wild, scattered literally across the globe, we needed a framework to allow remote monitoring, updating, and application deployment.  Arm Mbed Linux OS is a lightweight, secure, container-based operating system that meets these requirements.  It is currently still in Technical Preview, but is far enough along in development that it meets our project needs and is working great.  A total of 10 Raspberry Pi Endpoints were built and sent around the globe, with several across the United States, as well as Cambridge, Budapest, Delhi, Southern India, and one spare unit left over for local testing.

Turning to our Edge Nodes, these are the simplest component in our project’s infrastructure. These are 96Boards devices, chosen for their support and ease-of-use.  Linaro and the 96Boards team do an excellent job of building ready-made Debian images with updated kernels, applications, and drivers for their hardware, making for a great out-of-the-box experience. Two of these devices are currently provisioned, one in India and one in the United States, each serving their geographic region. The devices aggregate the IoT Endpoint data stream, convert it to the format needed by the Cloud Server, and publish the data to the Cloud.

Finally, the Arm-powered Cloud Server is an Ampere eMAG server, hosted by Packet.com. It is an enterprise-grade machine, and functions as the data warehouse for all of the IoT data, as well as a visual platform for charting and viewing the data in a time-series fashion thanks to InfluxDB and Grafana. Packet.com has datacenters around the world, and their native tooling and user interface make deploying Arm Servers quick and easy.

Now that the system architecture has been described, let’s take a look at the application architecture, and start to dissect how data flows from the IoT Endpoints, to the Edge, to the Cloud. As mentioned, Mbed Linux OS is a container-based OS, which is to say that it is a minimal underlying operating system based on Yocto, providing a small, lightweight, secure foundation to which the Open Container Initiative (OCI) “RunC” container engine is added.  RunC can launch OCI compliant containers built locally on your laptop, then pushed to the Endpoint via the Mbed Linux tooling, no matter where the device is located.  In our particular case, we chose a small Alpine Linux container, added Python, added the Sparkfun libraries for the sensors, and created a small startup script to begin reading data from the sensors when the container starts.  The container also has an MQTT broker in it, which is responsible for taking that sensor data, turning it into a small JSON snippet, and publishing it to a specific known location (the Edge Server).

The Edge Servers are a more traditional Debian operating system, with Python installed as well.  There is a Python script running as a daemon that captures and parses the incoming MQTT from IoT Endpoints, converts it to an InfluxDB formatted query, and publishes it to the specified Influx database, which is running on the Ampere eMAG Cloud Server.

Finally, the Cloud Server is an enterprise-grade Ampere eMAG Arm Server.  It is graciously hosted by the Works on Arm project at Packet.com, in their New Jersey datacenter. This server is also running Debian, and has InfluxDB and Grafana installed for storage and visualization of the data being sent to it from the Edge Nodes.  Thus, our IoT, Edge, and Server are all Arm-powered!

Construction Challenges

Building a container to hold our application did prove more challenging then anticipated, as a result of some needed functionality not provided by the ready-made Mbed Linux downloads. Normally, this could be easily solved by adding the desired packages to the Yocto build scripts and rebuilding from source…however, there is one additional and very unique quirk to this project: We have decided to exclusively use Arm-powered Windows on Snapdragon laptops to build the project!  These laptops are highly efficient, with all-day battery life and far better performance than previous generations offered.  One limitation however, is they are currently unable to run Docker, which we would need to re-build Mbed Linux from source.  Thus, instead of adding the necessary packages to Yocto and recompiling, we instead had to manually port Device Tree functionality, gain access to the GPIO pins on the Pi, enable the I2C bus and tooling, and finally expose that functionality from the host OS to the container, all by way of manually lifting from Raspbian.  Obviously, we placed this limitation upon ourselves, but it does demonstrate that there are still a few shortcomings in the developer experience on Arm.

A second valuable lesson learned is with the native Mbed tooling for initially deploying devices.  Provisioning and updating devices with Pelion Device Management is a straightforward process, except for one small but critical hiccup we experienced.  It is worth noting here again that Mbed Linux OS is in a Technical Preview status, and the feedback we were able to give to the Mbed team as a result of this process has been incorporated and will make the final product even better!  However, when following the documentation to provision devices for the first time, a Developer Certificate is issued. That certificate is only valid for 90 days, and after that time you can no longer push containers to a device in the field. That Certificate can certainly be updated via the re-provisioning process, but, you must be on the same network as the device in order to perform that action. Our devices are already out in the field, so that is not possible at this point.  Thus, we have a fleet of devices that cannot receive their intended application.  On the plus side, this exercise proved it’s worth by highlighting this point of failure, and resulted in the valuable documentation update so that your project can be a success!

Conclusion

In the end, we were able to successfully provision just a few devices that we still had local access to, and prove that the theory was sound and demonstrate a functional prototype at Arm TechCon!

Using a pair of freshly provisioned Raspberry Pi’s, the containerized application was pushed Over The Air to them, via the Mbed CLI.  Pelion showed the devices as Online, and the device and application logs in the Dashboard reported the container was started successfully.  Sure enough, on the Edge Node, data began streaming in, and the MQTT Broker began taking those transmissions, translating them to Influx, and sending them upstream to the Cloud Server.  Logged into Grafana running on the Cloud Server, that data could then be inspected and visualized.

Thus, while it wasn’t quite as geographically diverse as hoped, we did actually accomplish what we set out to do, which was build an end-to-end IoT, Edge, and Cloud infrastructure running entirely on Arm!  The data that is flowing is certainly just a minimal example, but as a proof-of-concept we can truthfully say that the design is valid and the system works!  Now, we’re excited to see what you can build to bring Simon Segar’s vision of the connected future to life!

Posted on Leave a comment

Arm Server Update, Summer 2019

It has been a while since our last Arm Server update, and as usual there has been a lot of changes, forward progress, and new developments in the ecosystem!

The enterprise Arm Server hardware is now mostly consolidated around the Cavium ThunderX2 and Ampere eMag products, available from Gigabyte, Avantek and Phoenics Electronics. Each can be purchased in 1U, 2U, and 4U configurations ready for the datacenter, and high performance developer workstations based on the same hardware are available, as well. Both of these solutions can be customized with additional RAM, storage, and networking, to best fit the intended workload.

Another option that exists, but is difficult to obtain in the United States, is the Huawei 1620, also known as the Kunpeng 920. These servers are also enterprise grade servers ready to be installed in a datacenter environment, typically in a 2U chassis with configurable memory and storage options. However, availability outside of Asia is limited, and new regulations may make importing them difficult.

While the Cavium, Ampere, and (potentially) Huawei servers are available as bare-metal options shipped directly to you for installation in your own datacenter, Amazon has also made significant progress over the past few months and is rapidly becoming the most popular Arm Server provider. They use their own Arm Server CPU called the Gravitron, that they use in their own proprietary AWS A1 ECS instances. This is quickly becoming the best way to deploy Arm Servers, as the entire system is in the Cloud and no hardware has to be purchased. They come in various sizes and price ranges, and experienced developers organizations who are familiar with the AWS system can simply pay by the hour for temporary workloads. For users who are less familiar with the ECS dashboard, less comfortable with the fluctuations in billing model, or prefer a fixed rate, we at miniNodes offer pre-configured Arm VPS servers in a range of sizes and prices, hosted atop the AWS platform.

Finally, the Edge of the network continues to be where a lot of innovation is occurring, and Arm Servers are a perfect fit for deplopyment as Edge Servers, due to their low power consumption, cost-effectiveness, and wide range of size and formats. The MacchiattoBin has been demonstrated running workloads in the base of windmills, the new SolidRun Clearfog ITX is promising to be a flexible solution, and the new Odroid N2 is an intersting device that has “enough” performance to satisfy a wide range of workloads that don’t need to always rely on the Cloud, and can instead deliver services and data to end-users (or other devices) faster by being located in closer proximity to where compute is needed.

As always, check back regularly for updates and Arm Server news, or follow us on Twitter where we share Arm related news on a daily basis!

Posted on Leave a comment

ARM Server Update, Summer 2018

Continuing our quarterly ARM Server update series, it is now Summer 2018 so it is time to review the ARM Server news and ecosystem updates from the past few months!  This blog series only covers the ARM Server highlights, but for more in-depth ARM Server news be sure to check out the Works on Arm Newsletter, delivered every Friday by Ed Vielmetti!

Looking at our recent blog posts, the most important headline seems to be the rumored exit from the business by Qualcomm.  Although, at the moment, this has not been confirmed, if true it would be a major setback for ARM Servers in the datacenter.  The Qualcomm Centriq had been shown to be very effective by CloudFlare for their distributed caching workload, and had been shown by Microsoft to be running a portion of the Azure workload as well.

However, just as Qualcomm is rumored to be exiting, Cavium has released the new ThunderX2 to general availability, and several new designs have now been shown and are listed for sale.  The ThunderX2 processor is a 32-core design that can directly compete with Xeons, and provides all of the platform features that a hyperscaler would expect.

Finally, in software news, Ubuntu has released it’s latest 18.04 Bionic Beaver release, which is an LTS version, thus offering 5 years of support.  As in the past, there is an ARM64 version of Ubuntu, which should technically work on any UEFI standard ARM Server.  Examples include Ampere X-Gene servers, Cavium ThunderX servers, Qualcomm, Huawei, HP Moonshot, and AMD Seattle servers.

As always, make sure to check back for more ARM Server and Datacenter industry news, or follow us on Twitter for daily updates on all things ARM, IoT, single board computers, edge computing, and more!

 

Posted on Leave a comment

Report: Qualcomm Looking to Exit ARM Server Processor Business

Recently, Bloomberg ran an article claiming that Qualcomm was seeking to close down or find a buyer for it’s ARM Server processor, the Centriq.  While the report has not been publicly confirmed by the company, if true, this would be welcome news to Cavium who just launched their ThunderX2 ARM Server processor.  Ampere could also benefit from this, as they are currently preparing to launch an updated X-Gene ARM Server processor based on the Applied Micro deisgn.

It would be a loss for the ARM Server ecosystem as a whole though, as the Centriq was well received in the press and reviews showed that the chip offered superior performance, lower power consumption, and excellent network throughput.

Here’s hoping this report is false!

 

Posted on Leave a comment

Report: Qualcomm Centriq TCO beats Intel

Tirias Research recently released a new Report detailing the Qualcomm Centriq Total Cost of Ownership versus an Intel Xeon x86 platform on a common workload, and the Qualcomm came out far ahead.  The full article is located here:  https://www.forbes.com/sites/tiriasresearch/2018/02/20/qdt-improved-server-tco/#3bbff2bc4675  The relevant piece is this:

Our TCO analysis demonstrated that using only one Qualcomm Centriq 2452 SoC per server chassis, a 12kW rack full of 36 46-core SoCs should show slightly better performance than a rack full of Intel Xeon Silver 4110 dual-socket server chassis, at only 51% of the power consumption. That’s similar performance with about half the power consumption.

 

Using two Qualcomm Centriq 2452 SoCs per server chassis in a 12kW rack should yield a little over double the performance of the dual-socket Intel Xeon Silver 4110 servers at 88% of the power consumption. A key factor is that only 35 of the Intel Xeon Silver 4110 systems can fit within the 12kW rack power budget. In this scenario, Qualcomm Centriq 2400 offers double the performance with less power consumption.

So, a single socket Centriq is essentially using half as much power for the exact same performance and workload, translating in to real savings.  And, there is room for performance improvement as well, by moving up to a dual socket design.  In that scenario, doubling the performance of the Xeon rack still results in a 12% power budget savings.  Double the performance and still drawing less power per rack, Qualcomm’s going to be challenging Intel’s dominance in the datacenter.