ValueError: Masked arrays must be 1-D

The following code was written in python to generate a scatter plot of the test data after splitting it into test and train data. The data being read from a sample file was converted to matrix to be able to use the data for the the linear regression algorithm.

import numpy as np import matplotlib.pyplot as plt import pandas as pd data=pd.read_csv('data_1.csv',encoding='utf-8') print data.head() X_data=np.matrix(data['X1']) Y_data=np.matrix(data['Y1']) X_train,X_test,Y_train,Y_test=train_test_split(np.transpose(X_data),\ np.transpose(Y_data), test_size=0.3) plt.scatter(X_test, Y_test, color='black')

But using the same data for the scatter plot generated the following error.

ValueError: Masked arrays must be 1-D

This is because we are passing X_test and Y_test and matrices to scatter plot but which actually are one dimensional arrays. Thus to get around the problem we can cast the data into an numpy array.

plt.scatter(np.array(X_test), np.array(Y_test), color='black')

The above modification should stop the error from being thrown. The modified code will be.

import numpy as np import matplotlib.pyplot as plt import pandas as pd data=pd.read_csv('data_1.csv',encoding='utf-8') print data.head() X_data=np.matrix(data['X1']) Y_data=np.matrix(data['Y1']) X_train,X_test,Y_train,Y_test=train_test_split(np.transpose(X_data),\ np.transpose(Y_data), test_size=0.3) plt.scatter(np.array(X_test), np.array(Y_test), color='black')

ValueError: Found input variables with inconsistent numbers of samples

The following code is written in python to run the liner regression algorithm on a given set of data. Two columns were chosen, namely X1 and and Y1 were chosen on which linear regression was to be performed. The code used for the same was

import numpy as np from sklearn import linear_model from sklearn.model_selection import train_test_split import pandas as pd data=pd.read_csv('data_1.csv',encoding='utf-8') X_data=np.array(data['X1']) Y_data=np.array(data['Y1']) X_train,X_test,Y_train,Y_test=train_test_split(X_data,Y_data, test_size=0.3) print X_train.shape print Y_train.shape regr = linear_model.LinearRegression() regr.fit(X_train,Y_train)

On executing the above code the following error was encountered.

ValueError: Found input variables with inconsistent numbers of samples: [1, 1336]

The above error generally comes when the X and Y have different number of samples. But in this case the error appeared even though the number of samples in X_train and Y_train were same as shown in the output.

(1336,) (1336,)

The problem with the code is that we converted the given data into numpy arrays, but it was required to convert them to numpy matrix to be able to pass it to fit. Thus we modify the reading of data to

X_data=np.matrix(data['X1']) Y_data=np.matrix(data['Y1'])

While doing the train test split we will need to transpose the matrix

X_train,X_test,Y_train,Y_test=train_test_split(np.transpose(X_data),\ np.transpose(Y_data), test_size=0.3)

Passing these test data to the fit function should not throw the value error encountered previously. The modified code thus would be

import numpy as np from sklearn import linear_model from sklearn.model_selection import train_test_split import pandas as pd data=pd.read_csv('data_1.csv',encoding='utf-8') print data.head() X_data=np.matrix(data['X1']) Y_data=np.matrix(data['Y1']) X_train,X_test,Y_train,Y_test=train_test_split(np.transpose(X_data),\ np.transpose(Y_data), test_size=0.3) print X_train.shape print Y_train.shape regr = linear_model.LinearRegression() regr.fit(X_train,Y_train)