SemusiSDK Accuracy - A Comparative Analysis

Semusi has been developing cutting-edge Machine Learning solutions with its SemusiSDK for Human Activity Recognition, Demographics recognition, Place labeling and Interest detection. As our systems have matured, we are now benchmarking our solutions against popular Android applications in the market.

This benchmark is against “Moves”, a popular pedometer applications that measures the steps taken by a user and the duration the user has been walking. We asked 6 people - 3 females and 3 male in their mid-20's, to walk 500 steps. The people were between 5 feet 3 inches to 6 feet in height and between 60-80kg in weight. We used 4 high- and low-end devices held in hand, which took 4.47 minutes of walking time. The GPS in all devices was turned off. The devices used were -

1. Samsung S4
2. Samsung S2
3. Moto G
4. Micromax A21

Summary:

Counting Steps -

As we can see from the graphs, SemusiSDK performs with 63.39% better accuracy on average in counting steps than Moves. SemusiSDK logged 323.5 steps on average, as against 182.58 by Moves.

SemusiSDK is 63.39% more accurate in the best case when using Micromax A21. SemusiSDK logged 285 steps on average, as against 104.33 by Moves.

SemusiSDK performs 59.80% better for females and 25.48% better for males than Moves for counting steps. SemusiSDK logged 340.83 steps on average for females and 306.17 for males, as against 137 and 228.17 respectively by Moves.

Counting Walking Duration -

As can be seen, SemusiSDK is 3.67x better on average for calculating minutes walked than Moves. SemusiSDK logged 3.21 minutes on average, as against 0.69 by Moves.

In the best case, SemusiSDK is 9.18x better when calculating minutes walked than Moves on Samsung Galaxy S4. SemusiSDK logged 3.39 minutes on average, as against 0.33 by Moves.

As in the plots, we can see that SemusiSDK 5.38x better for calculating minutes walked for females and 2.56x better for males than Moves. SemusiSDK logged 3.45 minutes for females and 2.97 minutes for males on average, as against 0.54 and 0.83 respectively by Moves.

 Figure 1: Average steps calculated by SemusiSDK and Moves, and the actual steps walked

 Figure 2: Average steps using Samsung Galaxy S4 over 3 Females, 3 Males

 Figure 3: Average steps using Samsung Galaxy S2 over 3 Females, 3 Males

 Figure 4: Average steps using Moto G over 3 Females, 3 Males

 Figure 5: Average steps using Micromax A21 over 3 Females, 3 Males

 Figure 6: Average steps calculated by SemusiSDK and Moves for 3 Females

 Figure 7: Average steps calculated by SemusiSDK and Moves for 3 Males

 Figure 8: Average minutes walked as calculated by SemusiSDK and Moves, and the actual steps walked

 Figure 9: Average minutes walked using Samsung Galaxy S4 over 3 Females, 3 Males

 Figure 10: Average minutes walked using Samsung Galaxy S2 over 3 Females, 3 Males

 Figure 11: Average minutes walked using Moto G over 3 Females, 3 Males

 Figure 12: Average minutes walked using Micromax A21 over 3 Females, 3 Males

 Figure 13: Average minutes walked by SemusiSDK and Moves for 3 Females

Figure 14: Average minutes walked by SemusiSDK and Moves for 3 Males

Table 1

The actual logged data is as below.

Actual Steps – 500

Actual Minutes – 4.47

User	Device	SemusiSDK Recorded Steps	SemusiSDK Recorded Minutes	Moves Recorded Steps	Moves Recorded Minutes
Female 1	Samsung Galaxy S4	507	3.37	383	0
	Samsung Galaxy S2	259	2.92	0	0
	Moto G	53	1.17	60	0
	Micromax A21	310	3.9	5	0
Female 2	Samsung Galaxy S4	794	4.9	463	0
	Samsung Galaxy S2	380	4.5	0	0
	Moto G	281	5.22	18	0
	Micromax A21	310	3.9	5	0
Female 3	Samsung Galaxy S4	434	2.78	440	0
	Samsung Galaxy S2	295	2.81	270	2
	Moto G	265	3.32	0	0
	Micromax A21	202	2.55	0	4.47
Male 1	Samsung Galaxy S4	335	3.05	319	0
	Samsung Galaxy S2	273	2.6	101	2
	Moto G	278	3.29	0	0
	Micromax A21	225	2.55	0	0
Male 2	Samsung Galaxy S4	554	3.38	469	2
	Samsung Galaxy S2	232	2.78	163	0
	Moto G	293	3.94	176	0
	Micromax A21	329	2.69	435	0
Male 3	Samsung Galaxy S4	274	2.88	390	0
	Samsung Galaxy S2	290	2.58	270	2
	Moto G	257	3.31	234	4
	Micromax A21	334	2.5995	181	0