All posts in: Swift

Today, 1th of June Google brain team committed new code in public.
There are some interesting points:
1) High level APIs will be presented as a separate SwiftPM package under github.com/tensorflow.

High level APIs were added earlier purely to explore the programming model, not to be usable by anyone. Having high level APIs be part of the stdlib module conveys a wrong message for beta testers, and it has been confusing ever since our open source release.

2) Supporting Python code is one of priority:

  • Improved Python diagnostics related to member access.
  • Improved Python C API functions for binary arithmetic operations.

3) Improved cross-device sends and receives support.

4) Lots of work done around supporting generic @dynamicCallable methods.

5) Deprecated a.dot(b) and to matmul(a, b).

Before reading this article, I recommend to have a look at my previous article on TensorFlowKit and like my repository.

When I started working in the field of machine learning, it was quite difficult to move to vectors and spaces from objects and their behavior. At first it was rather complicated to wrap my head around all that, and most processes did not seem obvious and clear at once. That’s the reason why I did my best to visualize everything I did in my groundwork:  I used to create 3D models, graphs, diagrams, figures, etc.

When speaking about efficient development of machine learning systems, usually such problems as learning speed control, learning process analysis, gathering various learning metrics, and others are mentioned. The major difficulty is that we (people) use 2D and 3D spaces to describe various processes that take place around us. However, processes within neural networks lay in multidimensional spaces, and that makes them rather difficult to understand. Engineers all around the world understand this problem and try to develop various approaches to the visualization or conversion of multidimensional data into simpler and more understandable forms.

There are separate communities dedicated to solving such problems, for example, Distill, Welch Labs, 3Blue1Brown.

Read More

Please, read my previous post about Swift & TensorFlow

MNIST

I took “Hello World!”  in the universe of neural networks as an example, a task for systematization of MNIST images. MNIST dataset includes thousands of images of handwritten numbers, the size of each image is 28×28 pixels. So, we have ten classes that are neatly divided into 60 000 images for educating and 10 000 images for testing. Our task is to create a neural network that is able to classify an image and determine the class it belongs to (out of 10 classes).

Read More

Each machine learning task is related with big amount of data. Analyzing a network is a complex and confusing task. To resolve that issue, Google announced launch of visualization tools called TensorBoard.

Currently that is the most useful source-code tool. Unfortunately that tool works only with TensorFlow library from the box. There is no way to feed it with json or xml logs.

Deepening  into a self-written neural network you can’t avoid any data-visualization task. For that reason you can use Tensorboard from C/C++/Java or Swift application.

How to do that, I will describe further.

Read More

Forward propagation as well as backpropagation leads to some operations on matrixes. The most common one is a matrix multiplication. In order to perform matrix multiplication in reasonable time you will need to optimise your algorithms.

There is a simple way to do it on macOS by means of their Accelerate Framework . Actually this is an umbrella framework for vector-optimized operations:

  • vecLib.framework – Contains vector-optimized interfaces for performing math, big-number, and DSP calculations, among others.
  • vImage.framework – Contains vector-optimized interfaces for manipulating image data.

Cblas_sgemm function can help you reach really hight performance.

Actually, vecLib  is only a ported version of two libs BLAS and LAPACK.

cblas.h and vblas.h are the interfaces to Apple’s implementations of BLAS. You can find reference documentation in BLAS. Additional documentation on the BLAS standard, including reference implementations, can be found on the web starting from the BLAS FAQ page at these URLs: http://www.netlib.org/blas/faq.html and http://www.netlib.org/blas/blast-forum/blast-forum.html.

clapack.h is the interface to Apple’s implementation of LAPACK. Documentation of the LAPACK interfaces, including reference implementations, can be found on the web starting from the LAPACK FAQ page at this URL: http://netlib.org/lapack/faq.html

This is a good way to combine your code with C++ library on Linux and macOS platforms.
Read More