Number of digits displayed/used - Knowledge Base from Adept Scientific

Knowledge Base > Browse Categories > Quality Analyst

Number of digits displayed/used

Email This Article

To email this article enter and address below and click on the "Email" Article button.

Other resources from Adept Scientific for: Quality Analyst

	NWA Quality Analyst Product Information
	System Requirements
	Download NWA Quality Analyst resources from our Download Centre
	Buy NWA Quality Analyst online from our Webstore
	Read NWA Quality Analyst technical articles from our KnowledgeBase
	Find out about NWA Quality Analyst Training

Last Modified:	3rd Jan 2013
Category:	Quality Control and Management > Quality Analyst
Version:	-

Article Ref.:	1FAE
»Return to previous search
»Print friendly version of this article.
1 person has found this article useful.

Both Quality Monitor and Quality Analyst use Single Precision maths ( 7 significant digits ). However, Quality Analyst may display more digits but anything after 7 significant digits in a calculation is questionable.

Also there are likely to be problems charting to 7 digits. If, for example your data is in the range 0.1234569 to 0.1234561, remember to check the scale and label the Y axis appropriately. So you may need to note the multiplication factor in the axis label.

Related Articles