Human Activity Recognition

Using smartphone sensors data, we should predict the activity that the person is performing

For instance:(Walking, Standing, Sitting, Laying)

It has two sensors

Accelerometer: It measures the acceleration

Gyroscope: It measures the Angular velocity

You may wonder why this is useful?

It is used to predict how many calories we have burnt based on our daily activity and also measures the human heart rates.

According to Wikipedia,

A smartwatch is a wearable computer in the form of a watch; modern smartwatches provide a local touchscreen interface for daily use, while an associated smartphone app provides for management and telemetry (such as long-term biomonitoring).

Let’s see the problem mathematically,

Most of the accelerometer is called Tri-axial if it measures 3 axes (x,y,z). Similarly, Gyroscope measures 3 axes.

We have six-time series data as input and need to predict one of the six activity

Obviously, It looks like a Multiclass classification problem.

Performance metric:

1.Accuracy

2.Confusion matrix

3. Multiclass log loss

Accelerometer and Gyroscope readings are taken from 30 volunteers(referred to as subjects) while performing the following 6 Activities.

1. Walking
2. Walking Upstairs
3. Walking Downstairs
4. Standing
5. Sitting
6. Lying.

• Readings are divided into a window of 2.56 seconds with 50% overlapping.
• Accelerometer readings are divided into gravity acceleration and body acceleration readings, which have x,y, and z components each.
• Gyroscope readings are the measure of angular velocities that have x,y, and z components.
• Jerk signals are calculated for BodyAcceleration readings.
• Fourier Transforms are made on the above time readings to obtain frequency readings.
• Now, on all the base signal readings., mean, max, mad, sma, AR coefficient, energy bands, entropy, etc., are calculated for each window.
• We get a feature vector of 561 features and these features are given in the dataset.
• Each window of readings is a data point of 561 features.

Let’s jump into the data set we have,

1. Need to build a 6 class classifier
2. breaking the time series up into each window and Convert into a vector (first vector)after applying a filter on top of it.
3. for the second vector, we need to do overlapping the previous window by half and so on we need to calculate the vector for each window.

Credits: Google

Original data:

Accelerometer : x,y,z

gyroscope : x,y,z

totally we have six-time series data

Domain expert data:

Domain expert has played a very import role in featuring the data.

This case study trade-off between

1. a human expert engineering feature

Here we are building a classical ML model

2.Raw Time series data

Here we are building a deep learning model. Because we know RNN can handle time-series data

will see how classified the model would be as compared to the deep learning model

If you want to solve problems like this you should consult with a domain expert in a particular field.

Or else, we can use a deep learning model to solve this problem

The next Question will be,

How do we break the data?

Ok, let’s do some EDA using this data set

Observations:

We have got the almost same number of reading from all the subjects

Observations:

This data is mostly balanced

Featuring Engineering from Domain Knowledge

• Static and Dynamic Activities
• In static activities (sit, stand, lie down) motion information will not be very useful.
• In the dynamic activities (Walking, Walking Upstairs, Walking Downstairs) motion info will be significant.

Stationary and Moving activities are completely different

The magnitude of acceleration can separate it well

Observations:

• If tAccMean is < -0.8 then the Activities are either Standing or Sitting or Laying.
• If tAccMean is > -0.6 then the Activities are either Walking or WalkingDownstairs or WalkingUpstairs.
• If tAccMean > 0.0 then the Activity is WalkingDownstairs.
• We can classify 75% of the Activity labels with some errors.

Position of Gravity Acceleration Components also matters

Observations:

• If angleX, gravityMean > 0 then Activity is Laying.
• We can classify all data points belonging to Laying activity with just a single if-else statement.

Apply t-see on the data

We have converted 561 dimensions into 2 dimension

Observations:

The first thing we should notice that most of the points are fairly separated except for sitting and standing.

The challenge is to separate sitting and standing features

Also, I would like to know what changes will happen if perplexity value changes

Observation:

Each data point is well separated except standing and sitting.

Observation:

Even changing More perplexity output remains the same.

Building a Classical Machine learning Models

Here we have used 561 expert engineered features,

Observation:

12% of sitting points are classified as a standing

Conclusion

We can choose Logistic regression or Linear SVC or RBF SVM.

LSTM BASED DL MODEL

Now we can implement a simple deep learning model that automatically can take engineered features from Raw time series data.

Let use a simple LSTM model

You can see, still, we have confusion between sitting and standing

Our handcrafted expert engineered features have given 96%

Having said that, the accuracy in which we get here 90.97~91% without any domain knowledge by just training a simple LSTM model.

Note:

This hands-on case study had done with the help of the Applied AI team

Credit: AAIC private limited.