Mixed Precision Tuning with Salsa

Nasrine Damouche and Matthieu Martel

LAMPS Laboratory, University of Perpignan, France

Keywords:

Mixed Precision, Numerical Accuracy, Program Transformation, Floating-point Arithmetic.

Abstract:

Precision tuning consists of ﬁnding the least ﬂoating-point formats enabling a program to compute some

results with an accuracy requirement. In mixed precision, this problem has a huge combinatory since any

value may have its own format. Precision tuning has given rise to the development of several tools that aim at

guarantying a desired precision on the outputs of programs doing ﬂoating-point computations, by minimizing

the initial, over-estimated, precision of the inputs and intermediary results. In this article, we present an

extension of our tool for numerical accuracy, Salsa, which performs precision tuning. Originally, Salsa is

a program transformation tool based on static analysis and which improves the accuracy of ﬂoating-point

computations. We have extended Salsa with a precision tuning static analysis. We present experimental results

showing the efﬁciency of this new feature as well as the additional gains that we obtain by performing Salsa’s

program transformation before the precision tuning analysis. We experiment our tool on a set of programs

coming from various domains like embedded systems and numerical analysis.

1 INTRODUCTION

Precision tuning consists of ﬁnding the least ﬂoating-

point formats enabling a program to compute some

results with an accuracy requirement. This problem

has many applications, such as image transmission

protocols from observation satellites to earth as de-

ﬁned by the European Cooperation for Space Stan-

dardization initiative

. Precision tuning allows com-

pilers to select the most appropriate formats (for ex-

ample IEEE754 (ANSI/IEEE, 2008) half, single, dou-

ble or quadruple formats (ANSI/IEEE, 2008; Muller

et al., 2010)) for each variable. It is then possible to

save memory, reduce CPU usage and use less band-

width for communications whenever distributed ap-

plications are concerned. So, the choice of the best

ﬂoating-point formats is an important compile-time

optimization in many contexts. Precision tuning is

also of great interest for the ﬁxed-point arithmetic for

which it is important to determine data formats, for

example in FPGAs (Gao et al., 2013; Martel et al.,

2014). In mixed precision, i.e. when every variable

or intermediary result may have its own format, pos-

sibly different from the format of the other variables,

this problem has a huge combinatory.

Recently, several approaches and tools have

been proposed for precision tuning, based on dyna-

http://ecss.nl/

mic (Lam et al., 2013; Rubio-Gonzalez et al., 2013)

or static (Martel, 2017) analysis. In this article, we

introduce the precision tuning analysis that we have

implemented in Salsa (Damouche et al., 2017a), a

tool dedicated to numerical accuracy. Salsa is an au-

tomatic tool to improve the accuracy of the ﬂoating-

point computations done in numerical codes. Based

on static analysis methods by abstract interpretation,

Salsa takes as input an original program, applies to

it a set of transformations and then generates an opti-

mized program which is more accurate than the initial

one. The original and the transformed programs are

written in the same imperative language.

In this article, we are motivated by the need for

an efﬁcient precision with a better accuracy of varia-

bles of programs. Our precision tuning analysis (Mar-

tel, 2017) combines a forward and a backward static

analysis, done by abstract interpretation (Cousot and

Cousot, 1977). We express the forward and backward

analysis as a set of constraints made of propositional

logic formulas and relations between afﬁne expres-

sions over integers (and only integers). These con-

straints can be easily checked by a SMT solver (we

use Z3 in practice (de Moura and Bjørner, 2008)).

We show how we can combine mixed-precision

tuning of ﬂoating-point programs with the impro-

vement of the numerical accuracy done by program

transformation in Salsa. The study presented in this

Damouche, N. and Martel, M.

Mixed Precision Tuning with Salsa.

DOI: 10.5220/0006915500470056

In Proceedings of the 8th International Joint Conference on Pervasive and Embedded Computing and Communication Systems (PECCS 2018), pages 47-56

ISBN: 978-989-758-322-3

article use a benchmark of programs coming from

embedded systems and numerical analysis. We focus

in basic computation bricks such as PID controllers,

matrix-vector product or polynomial evaluation. We

consider several precisions (half, single double pre-

cision (ANSI/IEEE, 2008)), We show that if we ini-

tially set all the variables at a given precision and if

we require that the result of the computation has half

the precision of the inputs (e.g input in double pre-

cision and result in single precision) then our tuning

precision module reduce the format of the inputs by

an average factor of 60% This factor is even impro-

ved if we transform the original program in order to

improve its numerical accuracy. Our experiments also

show that the tuning analysis time is very short which

is important in order to embed it inside a compiler.

In this article, we focus on the accuracy of com-

putations and we omit other problems related to

runtime-errors (Barr et al., 2013; Bertrane et al.,

2011). In particular, overﬂows are not considered.

In practice, a static analysis computing the ranges of

the variables and rejecting programs which possibly

contain overﬂows is done before our precision tuning

analysis.

This article is organized as follows. In Section 2,

we give a brief description of the ﬂoating-point

arithmetic and we introduce the former work on

mixed-precision. We present in Section 3 the Salsa

tool and the precision tuning analysis implemented

in Salsa. In Section 4, we illustrate our approach

on various programs and then we discuss the results

obtained. Conclusion and future work are given in

Section 5.

2 PRELIMINARY ELEMENTS

In this section we introduce some background mate-

rial. A brief overview of the IEEE754 Standard is

given in Section 2.1 and related work is discussed in

Section 2.2.

2.1 Elements of Floating-point

Arithmetic

We introduce here some elements of ﬂoating-point

arithmetic (ANSI/IEEE, 2008; Muller et al., 2010).

First of all, a ﬂoating-point number x in base β is de-

ﬁned by

x = s ·(d

...d

p−1

) · β

= s · m · β

e−p+1

(1)

where s ∈ {−1,1} is the sign, m = d

.. . d

p−1

is the

signiﬁcand, 0 ≤ d

< β, 0 ≤ i ≤ p − 1, p is the preci-

sion and e is the exponent, e

min

≤ e ≤ e

max

Format Name p e bits e

min

max

Binary16 Half prec. 11 5 −14 +15

Binary32 Single prec. 24 8 −126 +127

Binary64 Double prec. 53 11 −1122 +1223

Binary128 Quadruple prec. 113 15 −16382 +16383

Figure 1: Basic binary IEEE754 formats.

A ﬂoating-point number x is normalized whene-

ver d

6= 0. Normalization avoids multiple repre-

sentations of the same number. The IEEE754 Stan-

dard also deﬁnes denormalized numbers which are

ﬂoating-point numbers with d

= d

= . .. = d

= 0,

k < p − 1 and e = e

min

. Denormalized numbers make

underﬂow gradual (Muller et al., 2010). The IEEE754

Standard deﬁnes binary formats (with β = 2) and de-

cimal formats (with β = 10). In this article, without

loss of generality, we only consider normalized num-

bers and we always assume that β = 2 (which is the

most common case in practice). The IEEE754 Stan-

dard also speciﬁes a few values for p, e

min

and e

max

which are summarized in Figure 1. Finally, special

values also are deﬁned: nan (Not a Number) resulting

from an invalid operation, ±∞ corresponding to over-

ﬂows, and +0 and −0 (signed zeros).

The IEEE754 Standard also deﬁnes ﬁve rounding

modes for elementary operations over ﬂoating-point

numbers. These modes are towards −∞, towards +∞,

towards zero, to the nearest ties to even and to the ne-

arest ties to away and we write them ◦

−∞

, ◦

+∞

, ◦

◦

∼

and ◦

∼

, respectively. The semantics of the ele-

mentary operations  ∈ {+, −, ×, ÷} is then deﬁned



◦

= ◦( f

 f

) (2)

where ◦ ∈ {◦

−∞

,◦

+∞

,◦

∼

,◦

∼

} denotes the roun-

ding mode. Equation (2) states that the result of a

ﬂoating-point operation 

◦

done with the rounding

mode ◦ returns what we would obtain by performing

the exact operation  and next rounding the result

using ◦. The IEEE754 Standard also speciﬁes how

the square root function must be rounded in a similar

way to Equation (2) but does not specify the roundoff

of other functions like sin, log, etc.

We introduce hereafter two functions which com-

pute the unit in the f irst place and the unit in the last

place of a ﬂoating-point number. These functions are

used further in this article to generate constraints en-

coding the way roundoff errors are propagated throug-

hout computations. The ufp of a number x is

ufp(x) = min



i ∈ N : 2

i+1

> x



= blog

(x)c . (3)

The ulp of a ﬂoating-point number which signiﬁcand

has size p is deﬁned by

ulp(x) = ufp(x) − p + 1 . (4)

PEC 2018 - International Conference on Pervasive and Embedded Computing

The ufp of a ﬂoating-point number corresponds to the

binary exponent of its most signiﬁcant digit. Conver-

sely, the ulp of a ﬂoating-point number corresponds

to the binary exponent of its least signiﬁcant digit.

Note that several deﬁnitions of the ulp have been gi-

ven (Muller, 2005).

2.2 Related Work

Several approaches have been proposed to determine

the best ﬂoating-point formats as a function of the ex-

pected accuracy on the results. Darulova and Kuncak

use a forward static analysis to compute the propaga-

tion of errors (Darulova and Kuncak, 2014). If the

computed bound on the accuracy satisﬁes the post-

conditions then the analysis is run again with a smal-

ler format until the best format is found. Note that

in this approach, all the values have the same format

(contrarily to our framework where each control-point

has its own format). While Darulova and Kuncak de-

velop their own static analysis, other static techniques

(Goubault, 2013; Solovyev et al., 2015) could be used

to infer from the forward error propagation the suita-

ble formats. Chiang et al. (Chiang et al., 2017) have

proposed a method to allocate a precision to the terms

of an arithmetic expression (only). They use a for-

mal analysis via Symbolic Taylor Expansions and er-

ror analysis based on interval functions. In spite of

our linear constraints, they solve a quadratically con-

strained quadratic program to obtain annotations.

Other approaches rely on dynamic analysis. For

instance, the Precimonious tool tries to decrease the

precision of variables and checks whether the accu-

racy requirements are still fulﬁlled (Nguyen et al.,

2016; Rubio-Gonzalez et al., 2013). Lam et al in-

strument binary codes in order to modify their preci-

sion without modifying the source codes (Lam et al.,

2013). They also propose a dynamic search method to

identify the pieces of code where the precision should

be modiﬁed.

Finally other work focus on formal methods and

numerical analysis. A ﬁrst related research direction

concerns formal proofs and the use of proof assistants

to guaranty the accuracy of ﬁnite-precision computa-

tions (Boldo et al., 2015; Harrison, 2007; Lee et al.,

2018). Another related research direction concerns

the compile-time optimization of programs in order

to improve the accuracy of the ﬂoating-point compu-

tation in function of given ranges for the inputs, wit-

hout modifying the formats of the numbers (Damou-

che et al., 2017a; P. Panchekha and Tatlock, 2015).

3 THE SALSA TOOL

In this section, we introduce our tool, Salsa, for nu-

merical accuracy optimization by program transfor-

mation. Section 3.1 presents the tool in general and,

in Section 3.2, we describe the module dedicated to

precision tuning.

3.1 Overview of Salsa

Salsa is a tool that improves the numerical accu-

racy of programs based on ﬂoating-point arithme-

tic (Damouche and Martel, 2017). It reduces partly

the round-off errors by automatically transforming

C-like programs in a source to source manner. We

have deﬁned a set of intraprocedural transformation

rules (Damouche et al., 2016a) like assignments, con-

ditionals, loops, etc., and interprocedural transforma-

tion rules (Damouche et al., 2017b) for functions and

other rules deal with arrays. Salsa relies on static

analysis by abstract interpretation to compute vari-

able ranges and round-off error bounds. It takes as

ﬁrst input ranges for the input variables of programs

id ∈ [a,b]. These ranges are given by the user or co-

ming from sensors. Salsa takes as second input a

program to be transformed. Salsa applies the requi-

red transformation rules and returns as output a trans-

formed program with better accuracy.

Salsa is composed of several modules. The ﬁrst

module is the parser that takes the original program

in C-like language with annotations, puts it in SSA

form and then returns its binary syntax tree. The se-

cond module consists in a static analyzer, based on ab-

stract interpretation (Cousot and Cousot, 1977), that

infers safe ranges for the variables and computes er-

rors bounds on them. The third module contains the

intraprocedural transformation rules. The fourth mo-

dule implements the interprocedural transformation

rules. The last module is the Sardana tool, that we

have integrated in our Salsa and call it on arithmetic

expressions in order to improve their numerical accu-

racy.

When transforming programs we build larger

arithmetic expressions that we choose to parse in a

different ways to ﬁnd a more accurate one. These

large expressions will be sliced at a given level of

the binary syntactic tree and assigned to intermedi-

ary variables named TMP. Note that the transformed

program is semantically different from the original

one but mathematically are equivalent. In (Damou-

che et al., 2017a), we have introduced a proof by in-

duction that demonstrate the correctness of our trans-

formation. In other words, we have proved that the

original and the transformed programs are equivalent.

Mixed Precision Tuning with Salsa

kp = 0.194 ;

kd = 0.028 ;

inv d t = 10.0;

m = [ -1.0 ,1 .0] ;

c = 0 .5;

eol d = 0.0 ;

t = 0 .0;

whi l e ( t < 50 0 . 0) {

e = c + (m * ( -1 .0)) ;

p = kp * e;

d = kd * invdt *( e +( eo ld *( - 1.0)) ) ;

r = p + d;

eol d = e ;

t = t + 1.0;

};

req u i r e _ a c c u rac y ( r ,1 1 );

m = [ -1.0 ,1 .0] ;

c = 0 .5;

eol d = 0.0 ;

t = 0 .0;

whi l e ( t < 50 0 . 0) {

e = c + (m * ( -1 .0)) ;

p = (e * 0 . 1 9 4 );

d = (( e * 0.2 8 ) + ( ( - 1 .0)*( eo ld *0.28) ) ) ;

r = (d + p );

eol d = e ;

t = t + 1.0;

};

req u i r e _ a c c u rac y ( r ,1 1 );

Figure 2: Initial (left) and transformed (right) PID Controllers.

For example, Figure 2 gives the initial program of

a PID Controller where all the input variables are de-

clared in single precision and the output variable is

in half precision. We give in Figure 2, the PID Con-

troller program after being transforming by our tool,

Salsa. For readability reasons, in Figure 2, we have

omitted the precision on the inputs and control points

of the programs. For example, when the program is

in single precision, all the control points are initiali-

zed to 24.

3.2 Precision Tuning in Salsa

In this section we introduce our module to determine

the minimal precision on the inputs and on the in-

termediary results of a program performing ﬂoating-

point computations in order to ensure a desired accu-

racy on the outputs. Our tool combines a forward and

a backward static analysis, done by abstract interpre-

tation (Cousot and Cousot, 1977). These static analy-

sis are done after the ﬁrst static analysis which com-

putes the ranges of the values at each control point.

The forward analysis is classical and propagates sa-

fely the errors on the inputs and on the results of

the intermediary operations in order to determine the

accuracy of the results. Next, based on the results

of the forward analysis and on assertions indicating

which accuracy the user wants for the outputs at some

control points, the backward analysis computes the

minimal precision needed for the inputs and interme-

diary results in order to satisfy the assertions.

We express the forward and backward transfer

functions as a set of constraints made of propositi-

onal logic formulas and relations between afﬁne ex-

pressions over integers (and only integers). Indeed,

these relations remain linear even if the analyzed pro-

gram contains non-linear computations. As a conse-

quence, these constraints can be easily checked by

a SMT solver (we use Z3 in practice (Barrett et al.,

2009; de Moura and Bjørner, 2008)). The advantage

of the solver appears in the backward analysis, when

one wants to determine the precision of the operands

of some binary operation between two operands a and

b, in order to obtain a certain accuracy on the result.

In general, it is possible to use a more precise a with

a less precise b or, conversely, to use a more precise

b with a less precise a. Because this choice arises at

almost any operation, there is a huge number of com-

binations on the admissible formats of all the data in

order to ensure a given accuracy on the results.

Example 3.1. For example, let us consider the pro-

gram of Figure 3 which implements a simple linear

ﬁlter. At each iteration t of the loop, the output

is computed as a function of the current input x

and of the values x

t−1

and y

t−1

of the former ite-

ration. Our program contains several annotations.

First, the statement require accuracy(y

,10)

on the last line of the code informs the system that

the programmer wants to have 10 accurate binary

digits on y

at this control point. In other words,

let y

= d

.. . d

· 2

for some n ≥ 10, the abso-

lute error between the value v that y

would have if

all the computations where done with real numbers

and the ﬂoating-point value ˆv of y

is less than 2

e−11

|v − ˆv| ≤ 2

e−9

. 

An abstract value [a, b]

represents the set of

ﬂoating-point values with p accurate bits ranging

from a to b. For example, in the code of Figure

3, the variables x

t−1

and x

are initialized to the

abstract value [1.0, 3.0]

thanks to the annotation

[1.0,3.0]#16. Let F

be the set of all ﬂoating-point

numbers with accuracy p. This means that, compared

PEC 2018 - International Conference on Pervasive and Embedded Computing

t−1

:= [1 .0 ,3. 0] #1 6;

t−1

:= 0. 0;

wh ile ( c) {

u : = 0. 3 * y

t−1

;

v : = 0. 7 * ( x

+ x

t−1

);

:= u + v;

t−1

:= y

;

};

re q ui re _ ac cu r ac y ( y

,10) ;

|9|

t−1

:= [1 .0 , 3.0 ]

|9|

; x

|9|

:= [1 .0 , 3.0 ]

|9|

;

|10|

t−1

:= 0.0

|10|

;

wh ile ( c) {

|10|

:= 0.3

|10|

t−1

;

|10|

:= 0.7

|11|

|10|

( x

|9|

|10|

|9|

t−1

);

|10|

:= u

|10|

;

|10|

t−1

:= y

|10|

; } ;

re q ui re _ ac cu r ac y ( y

,10) ;

vo la ti le ha lf x

t−1

, x

;

ha lf u , v , y

;

fl oat y

t−1

, t mp ;

t−1

:= 0. 0;

wh ile ( c) {

u : = 0. 3 * y

t−1

;

tmp := x

+ x

t−1

;

v : = 0. 7 * t mp ;

:= u + v;

t−1

:= y

;

};

|16|

t−1

:= [1 .0 , 3.0 ]

|16|

;

|16|

:= [1 .0 , 3.0 ]

|16|

;

|52|

t−1

:= 0.0

|52|

;

|52|

:= 0.3

|52|

t−1

;

|15|

:= 0.7

|52|

|15|

( x

|16|

t−1

);

|15|

:= u

|52|

|15|

;

|15|

t−1

:= y

|15|

;

|9|

t−1

:= [1 .0 , 3.0 ]

|9|

; x

|9|

:= [1 .0 , 3.0 ]

|9|

;

|8|

t−1

:= 0.0

|8|

;

|10|

:= 0.3

|8|

|10|

|8|

t−1

;

|10|

:= 0.7

|11|

|10|

( x

|9|

|10|

|9|

t−1

);

|10|

:= u

|10|

;

|10|

t−1

:= y

|10|

;

re q ui re _ ac cu r ac y ( y

,10) ;

Figure 3: Top left: Initial program. Top middle: Annotations after analysis. Top right: Final program with generated data

types Bottom left: Forward analysis (one iteration). Bottom middle: Backward analysis (one iteration).

to exact value v computed in inﬁnite precision, the

value ˆv = d

.. . d

· 2

of F

is such that |v − ˆv| ≤

e−p+1

. By deﬁnition, using the function ufp introdu-

ced in Equation (3), for any x ∈ F

the roundoff error

ε(x) on x is bounded by ε(x) < 2

ulp(x)

= 2

ufp(x)−p+1

Concerning the abstract values, intuitively we have

the concretization function

γ([a,b]

) = {x ∈ F

: a ≤ x ≤ b} . (5)

In our example, x

and x

t−1

belong to [1.0, 3.0]

which means, by deﬁnition, that these variables have

a value ˆv ranging in [1.0, 3.0] and such that the error

between ˆv and the value v that we would have in the

exact arithmetic is bounded by 2

ufp(x)−15

. Typically,

in this example, this information would come from

the speciﬁcation of the sensor related to x. By default,

the values for which no accuracy annotation is given

(for instance the value of y

t−1

in the example of

Figure 3) are considered as exact numbers rounded

to the nearest in double precision. In this format

numbers have 53 bits of signiﬁcand (see Figure 1).

The last bit being rounded, these numbers have 52

accurate bits in our terminology and, consequently,

by default values belong to F

in our framework.

Based on the accuracy of the inputs, our forward

analysis computes the accuracy of all the other varia-

bles and expressions. The program in the left bottom

corner of Figure 3 displays the result of the forward

analysis on the ﬁrst iteration of the loop. Let

→

⊕ denote

the forward addition, the result of x

t−1

has 16

accurate digits since

→

⊕(1.0#16,1.0#16) = 2.0#16,

→

⊕(1.0#16,3.0#16) = 4.0#17,

→

⊕(3.0#16,1.0#16) =

4.0#17,

→

⊕(3.0#16,3.0#16) = 6.0#16.

→

([1.0,3.0]#16, [1.0, 3.0]#16) = [2.0, 6.0]#16.

The backward analysis is performed after the for-

ward analysis and takes advantage of the accuracy re-

quirement at the end of the code (see the right bottom

corner of Figure 3 for an unfolding of the backward

analysis on the ﬁrst iteration of the loop). Since, in

our example, 10 bits only are required for y

, the re-

sult of the addition u+v also needs 10 accurate bits

only. By combining this information with the result

of the forward analysis, it is then possible to lower

the number of bits needed for one of the operands.

Let

←

⊕ be the backward addition.

Example 3.2. For example, for x

t−1

in the as-

signment of v, we have

←

⊕(2.0#10,1.0#16) = 1.0#8,

←

⊕(2.0#10,3.0#16) = −1.0#8,

←

⊕(6.0#10,1.0#16) =

5.0#9,

←

⊕(6.0#10,3.0#16) = 3.0#8. 

Conversely to the forward function, the interval

function now keeps the largest accuracy arising in the

computation of the bounds:

←

([2.0,6.0]#10, [1.0, 3.0]#16) = [1.0, 3.0]#9 .

By processing similarly on all the elementary ope-

rations and after computation of the loop ﬁxed point,

we obtain the ﬁnal result of the analysis displayed in

the top right corner of Figure 3. This information may

be used to determine the most appropriate data type

for each variable and operation, as shown in Figure 3.

To obtain this result we generate a set of constraints

corresponding to the forward and backward transfer

Mixed Precision Tuning with Salsa

functions for the operations of the program. There

exist several ways to handle a backward operation:

when the accuracy on the inputs x and y computed

by the forward analysis is too large wrt. the desired

accuracy on the result, one may lower the accuracy of

either x or y or both. Since this question arises at each

binary operation, we would face to a huge number of

combinations if we decided to enumerate all possibili-

ties. Instead, we generate a disjunction of constraints

corresponding to the minimization of the accuracy of

each operand and we let the solver search for a solu-

tion. The control ﬂow of the program is also enco-

ded with constraints. For a sequence of statements,

we relate the accuracy of the former statements to the

accuracy of the latter ones. Each variable x has three

parameters: its forward, backward and ﬁnal accuracy,

denoted acc

(x), acc

(x) and acc(x) respectively. We

must always have

0 ≤ acc

(x) ≤ acc(x) ≤ acc

(x) . (6)

For the forward analysis, the accuracy of some va-

riable may decrease when passing to the next state-

ment (we may only weaken the pre-conditions). Con-

versely, in the backward analysis, the accuracy of a gi-

ven variable may increase when we jump to a former

statement in the control graph (the post-conditions

may only be strengthened). For a loop, we relate the

accuracy of the variables at the beginning and at the

end of the body, in a standard way.

The key point of our technique is to generate sim-

ple constraints made of propositional logic formu-

las and of afﬁne expressions among integers (even if

the ﬂoating-point computations in the source code are

non-linear). A static analysis computing safe ranges

at each control point is performed before our accuracy

analysis. Then the constraints depend on two kinds

of integer parameters: the ufp of the values and their

accuracies acc

, acc

and acc. For instance, given

control points `

, `

and `

, the set C of constraints

generated for 3.0#16

1.0#16

, assuming that

we require 10 accurate bits for the result are:

C =











acc

) = 16, acc

) = 16,

= 2 − max(acc

) − 1, acc

)),

(1 − acc

)) = acc

) ⇒ i

= 1,

(1 − acc

)) 6= acc

) ⇒ i

= 0,

acc

) = r

− i

, acc

) = 10

acc

) = 1 −(2 − acc

)),

acc

) = 1 −(2 − acc

))











For the sake of conciseness, the constraints corre-

sponding to Equation (6) have been omitted in C.

For example, for the forward addition, the accuracy

acc

) of the result is the number of bits between

ufp(3.0 + 1.0) = 2 and the ufp u of the error which is

u = max



ufp(3.0) − acc

),ufp(1.0) − acc

)



+ i

= max



1 − acc

),0 − acc

)



+ i ,

where i = 0 or i = 1 depending on some condition

detailed later. The constraints generated for each kind

of expression and command are detailed in (Martel,

2017).

4 EXPERIMENTS

In this section, we illustrate the usefulness of our

techniques by its application to various programs co-

ming from embedded systems and numerical analy-

sis. We start by giving a brief description of each

program considered. Then, we show and discuss the

result obtained using our techniques.

• PID Controller: This program implements a

controller which aims at maintaining a physical

parameter at a speciﬁc value known as the set-

point (Damouche et al., 2017a) (see Figure 2),

• Odometry: This program consists of computing

the position (x,y) of a two wheeled robots by odo-

metry (Damouche et al., 2017a), i.e., from the ini-

tial position and from the speed of the wheels,

• Linear Filter: This program implements a li-

near ﬁlter in which the value of an output signal

is a linear combination of the values of the input

sampled signal at the last instants (see Figure 3),

• Linear Regression: This program implements

linear regression, a technique for modeling the re-

lationship between a scalar dependent variable y

and one or more independent variables denoted X,

• Horner: This program implements a polynomial

evaluation method and is applied to a polynomial

of degree 9 (Martel, 2017),

• FPTaylor: This program corrects sensor data in

control software, where the output of the sen-

sor is known to be between 1.001 and 2.0 by

using Taylor series to narrow the computed error

bounds (Damouche et al., 2016b; Solovyev et al.,

2015),

• Taylor Series: This program implements the

function

1−x

using Taylor’s series.

• Determinant: This program computes the de-

terminant Det(M) of a 3 × 3 matrix M (Martel,

2017),

• MatVectMul: This program computes a 3 × 3 ma-

trix vector product,

• MatMatMul: This program computes a 3 × 3 ma-

trix matrix product.

Figures 4, 5 and 6 summarize the results obtained

for ﬁnding a minimal ﬂoating-point formats by ensu-

PEC 2018 - International Conference on Pervasive and Embedded Computing

Initial #nbCPT Initial Prec. Tun. Improvement Prec. Tun. Improvement

Code #nbCP after trans. prec. of prog. % with Salsa %

PID Controller 42 34 1008 381 62.2 282 72.0

Odometry 68 70 1632 486 70.2 437 73.2

Linear Filter 30 24 720 278 61.3 255 64.5

Linear Regression 44 28 1056 585 55.4 365 65.4

Horner 16 16 384 166 56.7 166 56.7

FPTaylor 18 20 432 243 43.7 - -

Taylor Series 22 22 528 188 64.4 155 70.6

Determinant 72 60 1728 547 68.3 443 74.3

MatVectMul 160 160 3840 1570 59.1 1570 59.1

MatMatMul 200 200 4800 1990 58.5 1990 58.5

Figure 4: Measurements of the improvement of the required precision of programs in single precision.

Initial #nbCPT Initial Prec. Tun. Improvement Prec. Tun. Improvement

Code #nbCP after trans. prec. of prog. % with Salsa %

PID Controller 42 34 2226 914 58.9 659 70.4

Odometry 68 70 3604 1136 68.4 996 72.3

Linear Filter 30 24 1590 616 61.2 567 64.3

Linear Regression 44 22 2332 1157 50.4 703 69.8

Horner 16 16 848 316 57.4 316 57.4

FPTaylor 18 20 954 477 50.0 - -

Taylor Series 22 22 1166 409 64.9 337 71.1

Determinant 72 60 3816 1483 61.1 1223 67.9

MatVectMul 160 160 8480 3650 56.9 3650 56.9

MatMatMul 200 200 10600 4486 57.6 4486 57.6

Figure 5: Measurements of the improvement of the required precision of programs in double precision.

ring a desired accuracy on the outputs. In our experi-

ments, we observe two sides for each program:

• The ﬁrst side concerns the computation of the

number of control points of each program when

just using the precision tuning analysis (#nbCP)

and then by transforming it and applying the

mixed precision analysis using Salsa (#nbCPT).

• The Second side consists of measuring the pre-

cision on the output of programs when using the

precision tuning analysis (Prog. Prec. Tun.) and

then by using Salsa (Salsa Prec. Tun.).

Note that these experiments have been performed on

three different precision. More precisely, we have de-

clared all the input variables of programs in single,

double and quadruple precision and we have ﬁxed the

desired accuracy of the outputs of programs in half,

single and double precision, respectively.

For instance, if we take the PID Controller exam-

ple given in Figure 4, we initially have a precision

of 1008 bits (Initial Prec.). This number is obtai-

ned by multiplying the number of control points of

the program by the precision associated to its inputs.

In this case, 1008 corresponds to 42 multiplied by

24 bits (single precision). The initial precision of the

PID controller is reduced to 381 bits using only pre-

cision tuning analysis (Prog. Prec. Tun.) and to 282

bits while using Salsa (Salsa Prec. Tun.). The per-

centage of improvement is passed from to 62.2% to

72.0%. In addition, by transforming programs with

Salsa, the number of control points is generally de-

creased. In the case of the PID program, this num-

ber passed from 42 control points (#nbCP) to 34

(#nbCPT). This reduction is mainly done because of

the transformation applied when using Salsa. Note

that, our tuning precision techniques reduces the for-

mat of the inputs by an average factor of 60%. Re-

mark that for the FPTaylor example, our tool has fai-

led to solve it. This is why, we represent it with -.

Figure 8 compares the different results obtained

with our techniques. It synthesizes the results of ﬁgu-

res 4 to 6. We have drawn the values of the precision

corresponding to the initial program, to the program

using only tuning analysis and to the program trans-

formed by Salsa and this for the different programs

formats. More precisely, Figure 8 gives a comparison

between all the considered programs in single, dou-

ble and quadruple precision respectively correspon-

ding to program24, program53 and program113 on

the x-axis of Figure 8. The results show that for most

programs, we have that when using our tuning ana-

lysis, the precision is reduced by more than 50% like

the PID controller, odometry, ﬁlter, linear regression,

etc. In general, we remark that by applying Salsa,

Mixed Precision Tuning with Salsa

Initial #nbCPT Initial Prec. Tun. Improvement Prec. Tun. Improvement

Code #nbCP after trans. prec. of prog. % with Salsa %

PID Controller 42 34 4764 2103 55.6 1500 68.3

Odometry 68 70 7684 2586 66.3 2243 70.8

Linear Filter 30 24 3390 1370 59.5 1263 62.7

Linear Regression 44 28 4972 2433 51.0 1457 70.7

Horner 16 16 1808 796 55.9 796 55.9

FPTaylor 18 20 2034 - - - -

Taylor Series 22 22 2486 902 63.7 743 70.1

Determinant 72 60 8136 3571 56.1 1223 84.9

MatVectMul 160 160 18080 8290 54.1 8290 54.1

MatMatMul 200 200 22600 10054 55.5 10054 55.5

Figure 6: Measurements of the improvement of the required precision of programs in quadruple precision.

Single Prec. (24 bits) Double Prec. (53 bits) Quad. Prec (113 bits)

Code Prog. Exec. Time Salsa Exec. Time Prog. Exec. Time Salsa Exec. Time Prog. Exec. Time Salsa Exec. Time

PID Controller 0.060 0.048 0.056 0.040 0.044 0.028

Odometry 0.048 0.032 0.044 0.036 0.036 0.032

Linear Filter 0.028 0.024 0.032 0.028 0.036 0.028

Linear Regression 0.028 0.024 0.036 0.024 0.036 0.024

Horner 0.024 0.020 0.032 0.032 0.028 0.016

FPTaylor 0.020 0.016 0.036 0.032 0.040 0.036

Taylor Series 0.024 0.024 0.032 0.028 0.036 0.028

Determinant 0.052 0.036 0.044 0.040 0.052 0.048

MatVectMul 0.068 0.056 0.052 0.040 0.068 0.064

MatMatMul 0.072 0.056 0.080 0.068 0.068 0.052

Figure 7: Execution time measurements of programs.

the precision of programs is improved by an average

of about 10 additional percents compared to the di-

rect tuning analysis. Otherwise, in some cases, like

the program that computes the matrix vector product,

or the matrix matrix product, there is no more impro-

vement. These results are very representative since it

allows to decrease widely the precision of programs.

It will be more interesting if we could improve even

more the accuracy of programs by ﬁnding new techni-

ques and strategies. Indeed, in future work we plan

to improve Salsa for this situation by modifying our

program transformation in order to favor the precision

tuning analysis and not only the accuracy during the

rewriting of the expressions.

In the other side, we have measured the execution

time of programs. More precisely, for each program,

we measure the execution time required by the the

program performing just the precision tuning analysis

(Prog. Exec. Time) and then when applying Salsa

(Salsa Exec. Time ) and this for single, double and

quadruple precision. Figure 7 summarized the dif-

ferent results obtained. For instance, if we consider

the PID Controller program, it takes 0.060 seconds

for execution when using the precision tuning analy-

sis while it takes 0.048 seconds when applying Salsa.

For the other programs, the execution time is almost

always improved.

These results shown the efﬁciency and the useful-

ness of our techniques on saving memory space, redu-

cing the CPU usage, decreasing the bandwidth usage

and reducing the execution time of each program.

5 CONCLUSION

In this article, we have studied how to ﬁnd an efﬁ-

cient precision with a better accuracy of variables of

programs. The originality of the idea is to improve

the numerical accuracy, by using our tool Salsa, and

simultaneously minimize the precision on the outputs

of variables of programs using precision tuning analy-

sis. We have experimented our techniques to optimize

the precision and the numerical accuracy programs at

once. The experiments shown that the precision of

the programs has been minimized by an average fac-

tor of 60%. Also, we have shown that by optimizing

the precision and the accuracy, the execution time of

programs is improved.

In our future work, it would be interesting to ex-

tend our techniques to design efﬁcient heuristic reﬁ-

nement that automatically determines and speciﬁes a

least format and a precise range of precision for each

variable of the program. This allows one to further

reduce the used memory space, CPU and bandwidth.

PEC 2018 - International Conference on Pervasive and Embedded Computing

5000

10000

15000

20000

25000

PID24

PID53

PID113

Odo24

Odo53

Odo113

Filter24

Filter53

Filter113

Reg24

Reg53

Reg113

Horner24

Horner53

Horner113

FPTay24

FPTay53

FPTay113

Series24

Series53

Series113

Det24

Det53

Det113

MatVec24

MatVec53

MatVec113

MatMat24

MatMat53

MatMat113

Quadruple precision

Initial

Mixed

Salsa

Figure 8: Improvements of the required precision of programs in single, double and quadruple precision.

Another perspective is to apply our techniques to

other languages used for the design of critical embed-

ded systems. In particular, it would be of great interest

to have such a type system inside a language used to

build critical embedded systems such as the synchro-

nous language Lustre (Caspi et al., 1987). In this

context numerical accuracy requirements are strong

and difﬁcult to obtain.

REFERENCES

ANSI/IEEE (2008). IEEE Standard for Binary Floating-

Point Arithmetic. SIAM.

Barr, E. T., Vo, T., Le, V., and Su, Z. (2013). Automatic

detection of ﬂoating-point exceptions. In POPL ’13,

pages 549–560. ACM.

Barrett, C. W., Sebastiani, R., Seshia, S. A., and Tinelli,

C. (2009). Satisﬁability modulo theories. In Hand-

book of Satisﬁability, volume 185 of Frontiers in Ar-

tiﬁcial Intelligence and Applications, pages 825–885.

IOS Press.

Bertrane, J., Cousot, P., Cousot, R., Feret, J., Mauborgne,

L., Min

e, A., and Rival, X. (2011). Static analysis by

abstract interpretation of embedded critical software.

ACM SIGSOFT Software Engineering Notes, 36(1):1–

Boldo, S., Jourdan, J., Leroy, X., and Melquiond, G. (2015).

Veriﬁed compilation of ﬂoating-point computations.

J. Autom. Reasoning, 54(2):135–163.

Caspi, P., Pilaud, D., Halbwachs, N., and Plaice, J. (1987).

Lustre: A declarative language for programming syn-

chronous systems. In ACM Symposium on Principles

of Programming Languages, pages 178–188. ACM

Press.

Chiang, W., Baranowski, M., Briggs, I., Solovyev, A., Go-

palakrishnan, G., and Rakamaric, Z. (2017). Rigorous

ﬂoating-point mixed-precision tuning. In POPL, pa-

ges 300–315. ACM.

Cousot, P. and Cousot, R. (1977). Abstract interpretation: A

uniﬁed lattice model for static analysis of programs by

construction or approximation of ﬁxpoints. In Princi-

ples of Programming Languages, pages 238–252.

Damouche, N. and Martel, M. (2017). Salsa : An automa-

tic tool improve the accuracy of programs. In 6th In-

ternational Workshop on Automated Formal Methods,

AFM.

Damouche, N., Martel, M., and Chapoutot, A. (2016a).

Data-types optimization for ﬂoating-point formats by

program transformation. In International Conference

on Control, Decision and Information Technologies,

CoDIT 2016, Saint Julian’s, Malta, April 6-8, 2016,

pages 576–581.

Damouche, N., Martel, M., and Chapoutot, A. (2017a). Im-

proving the numerical accuracy of programs by auto-

matic transformation. STTT, 19(4):427–448.

Damouche, N., Martel, M., and Chapoutot, A. (2017b).

Numerical accuracy improvement by interprocedural

program transformation. In Stuijk, S., editor, Procee-

dings of the 20th International Workshop on Software

and Compilers for Embedded Systems, SCOPES 2017,

Sankt Goar, Germany, June 12-13, 2017, pages 1–10.

ACM.

Damouche, N., Martel, M., Panchekha, P., Qiu, C.,

Sanchez-Stern, A., and Tatlock, Z. (2016b). Toward

a standard benchmark format and suite for ﬂoating-

point analysis. In S. Bogomolov, M. Martel, P. P., edi-

tor, NSV, LNCS. Springer.

Darulova, E. and Kuncak, V. (2014). Sound compilation

Mixed Precision Tuning with Salsa

of reals. In Jagannathan, S. and Sewell, P., editors,

POPL’14, pages 235–248. ACM.

de Moura, L. M. and Bjørner, N. (2008). Z3: an efﬁcient

SMT solver. In TACAS, volume 4963 of LNCS, pages

337–340. Springer.

Gao, X., Bayliss, S., and Constantinides, G. A. (2013).

SOAP: structural optimization of arithmetic expressi-

ons for high-level synthesis. In International Con-

ference on Field-Programmable Technology, pages

112–119. IEEE.

Goubault, E. (2013). Static analysis by abstract interpreta-

tion of numerical programs and systems, and FLUC-

TUAT. In SAS, volume 7935 of LNCS, pages 1–3.

Springer.

Harrison, J. (2007). Floating-point veriﬁcation. J. UCS,

13(5):629–638.

Lam, M. O., Hollingsworth, J. K., de Supinski, B. R.,

and LeGendre, M. P. (2013). Automatically adapting

programs for mixed-precision ﬂoating-point compu-

tation. In Supercomputing, ICS’13, pages 369–378.

ACM.

Lee, W., Sharma, R., and Aiken, A. (2018). On automati-

cally proving the correctness of math.h implementati-

ons. PACMPL, 2(POPL):47:1–47:32.

Martel, M. (2017). Floating-point format inference in

mixed-precision. In Barrett, C., Davies, M., and Ka-

hsai, T., editors, NASA Formal Methods - 9th Interna-

tional Symposium, NFM 2017, volume 10227 of Lec-

ture Notes in Computer Science, pages 230–246.

Martel, M., Najahi, A., and Revy, G. (2014). Code size

and accuracy-aware synthesis of ﬁxed-point programs

for matrix multiplication. In Pervasive and Embedded

Computing and Communication Systems, pages 204–

214. SciTePress.

Muller, J.-M. (2005). On the deﬁnition of ulp(x). Technical

Report 2005-09, Laboratoire d’Informatique du Pa-

rall

elisme, Ecole Normale Sup

erieure de Lyon.

Muller, J.-M., Brisebarre, N., de Dinechin, F., Jeannerod,

C.-P., Lef

evre, V., Melquiond, G., Revol, N., Stehl

D., and Torres, S. (2010). Handbook of Floating-Point

Arithmetic. Birkh

auser Boston.

Nguyen, C., Rubio-Gonzalez, C., Mehne, B., Sen, K., Dem-

mel, J., Kahan, W., Iancu, C., Lavrijsen, W., Bailey,

D. H., and Hough, D. (2016). Floating-point precision

tuning using blame analysis. In Int. Conf. on Software

Engineering (ICSE). ACM.

P. Panchekha, A. Sanchez-Stern, J. R. W. and Tatlock, Z.

(2015). Automatically improving accuracy for ﬂo-

ating point expressions. In PLDI’15, pages 1–11.

ACM.

Rubio-Gonzalez, C., Nguyen, C., Nguyen, H. D., Demmel,

J., Kahan, W., Sen, K., Bailey, D. H., Iancu, C., and

Hough, D. (2013). Precimonious: tuning assistant for

ﬂoating-point precision. In Int. Conf. for High Perfor-

mance Computing, Networking, Storage and Analysis,

pages 27:1–27:12. ACM.

Solovyev, A., Jacobsen, C., Rakamaric, Z., and Gopala-

krishnan, G. (2015). Rigorous estimation of ﬂoating-

point round-off errors with symbolic taylor expansi-

ons. In FM’15, volume 9109 of LNCS, pages 532–

550. Springer.

PEC 2018 - International Conference on Pervasive and Embedded Computing