Comparison Tolerance: OCT

124

0

The value of this variable determines the maximum tolerance (how different the two

numbers must be to be considered unequal) when using any relational function and

at least one argument is a noninteger. For example, two numbers are considered un-

equal if the relative difference between the two numbers exceeds the comparison

tolerance value. The following illustration shows how the comparison tolerance

works with the relational functions:

Value of argument A

Real number line

The relationship of

any value (argument 6)

to argument A - ArB

- A>B

Note: The OCT function considers any number in decimal form a noninteger. For

example, 1000 is an integer and 1000. is a noninteger.

The value of the comparison tolerance variable also affects the floor and ceiling

functions. If an integer is in the range of the right argument plus or minus the

comparison tolerance, the integer is the result. For example:

[IC1ā+. , 03

1-2 , 98

I.. 2 , 9 6

Iā 3 1 0 3

2.98 + .03 = 3.01 (The integer 3 is in the range of

2.96 + .03 = 2.99

3.03 - -03 = 3 (The integer 3 is in the range of

3 2.98 ! .03.)

3

3 3.03 ? .03.)

3

rz. 04. 3.04 - .03 3.01

In a clear workspace, the comparison tolerance value is set to 1 E-13 (see

Chaptey 3 for an explanation of scaled representation).

124

0

The value of this variable determines the maximum tolerance (how different the two

numbers must be to be considered unequal) when using any relational function and

at least one argument is a noninteger. For example, two numbers are considered un-

equal if the relative difference between the two numbers exceeds the comparison

tolerance value. The following illustration shows how the comparison tolerance

works with the relational functions:

Value of argument A

Real number line

The relationship of

any value (argument 6)

to argument A - ArB

- A>B

Note: The OCT function considers any number in decimal form a noninteger. For

example, 1000 is an integer and 1000. is a noninteger.

The value of the comparison tolerance variable also affects the floor and ceiling

functions. If an integer is in the range of the right argument plus or minus the

comparison tolerance, the integer is the result. For example:

[IC1ā+. , 03

1-2 , 98

I.. 2 , 9 6

Iā 3 1 0 3

2.98 + .03 = 3.01 (The integer 3 is in the range of

2.96 + .03 = 2.99

3.03 - -03 = 3 (The integer 3 is in the range of

3 2.98 ! .03.)

3

3 3.03 ? .03.)

3

rz. 04. 3.04 - .03 3.01

In a clear workspace, the comparison tolerance value is set to 1 E-13 (see

Chaptey 3 for an explanation of scaled representation).