Looking at the report that Gartner did in 2022 regarding top technology trends, AI engineering represents an important pillar in the near future. It is composed of three core technologies: DataOps, MLOps and DevOps.The discipline’s main purpose is to develop AI models that can quickly and continuously provide business value. For instance, models that enable cross-functional collaboration, automation, data analysis, and machine learning. AI engineering shortens the development and deployment processes.
Charmed Kubeflow is a production-grade, end-to-end MLOps platform that translates steps in the data science workflow into Kubernetes jobs. It is one of the official distributions of the Kubeflow upstream project. Using it, data scientists and machine learning engineers benefit from having ML deployments that are simple, portable and scalable. It has capabilities that cover a wide range, from experimentation using Notebooks, to training using the Kubeflow Pipelines or tuning, using Katib.
Learn more about what is Kubeflow?
MindSpore is an open-source AI framework built for deep learning, training and inference, that is used for edge, mobile and cloud scenarios. It implemented automatic differentiation based on source transformation. This is a technology evolving from the functional programming framework and it supports automatic control flow.
Learn more aboutMindspore.
Charmed Kubeflow is an MLOps platform, composed of various open-source components. Jupyter Notebooks is one of the tools that are natively available. Having integration with an AI Framework enables data scientists and machine learning engineers to benefit from advanced training capabilities and a cost model-splitting strategy. It combines data parallelism, model parallelism and hybrid parallelism. It leads on the one hand to execution efficiency, at the pipeline, deep graph or on-device level, as well as development friendliness, because of the automatic differential, parallelisation and optimisation.
Overall, the integration between Charmed Kubeflow and MindSpore highlights the growing MLOps ecosystem that Canonical is working on building. Juggling between tools, especially when working with data can be easily challenging. Thus, charmed Kubeflow aims to solve this problem, by creating an environment that natively offers access to various AI/ML tools.
If you have not deployed Charmed Kubeflow, please follow the instructions from the Quickstart guide to deploy it. Please bear in mind that you need to run Ubuntu 20.04 or later with, a minimum of 32GB of free memory and 50GB of disk space. During the deployment process, you will
Once Charmed Kubeflow is deployed and all Kubeflow components have the active status, please follow the instructions below.
juju refresh jupyter-ui --channel=latest/edge
This step updates jupyter-ui to the latest version, so it includes MindSpore’s Notebook image.
After Kubeflow is installed, you need to do a simple configuration to access it. First configure the access address:
juju config dex-auth public-url=http://10.64.140.43.nip.io
juju config oidc-gatekeeper public-url=http://10.64.140.43.nip.io
Then configure the username and password for access:
juju config dex-auth static-username=admin
juju config dex-auth static-password=ucantseeme
Next, enter http://10.64.140.43.nip.io in the browser, and the user password you just set. Then you can see the complete Dashboard of Kubeflow.
In order to access MindSpore within Charmed Kubeflow, you will need to use the natively enabled Notebooks. On the Notebook tab, from the Charmed Kubeflow dashboard, select Create Notebook, enter the notebook name MindSpore, select jupyterlab for the image, use mindspore/jupyter-mindspore from the list, allocate appropriate CPU and content resources, and click Create.
In the Notebook interface, you can see the prompt that the creation is completed soon.
Click Connect, we can open the notebook in another browser tab page. Here we can use the existing notebook on MindSpore’s official website to upload the handwritten digit recognition notebook through the notebook page.
Click Execute, and you can see that the notebook is running directly without additional installation of MindSpore and Vision suites.
Once you have the MindSpore image natively enabled in your Notebook from Charmed Kubeflow, you can go ahead and start your AI/ML projects. Whereas MindSpore is designed to support a wide range of use cases, computer vision might be an interesting example.
At Canonical, the work of our teams is strongly embedded in the open source principles…
Welcome to the Ubuntu Weekly Newsletter, Issue 873 for the week of December 29, 2024…
Have WiFi troubles on your Ubuntu 24.04 system? Don’t worry, you’re not alone. WiFi problems…
The following is a post from Mark Shuttleworth on the Ubuntu Discourse instance. For more…
I don’t like my prompt, i want to change it. it has my username and…
Introduction: A Fragile Trust The Ruby ecosystem relies heavily on RubyGems.org as the central platform…