Removing nan values from an array – Dev

The best answers to the question “Removing nan values from an array” in the category Dev.

QUESTION:

I want to figure out how to remove nan values from my array. My array looks something like this:

x = [1400, 1500, 1600, nan, nan, nan ,1700] #Not in this exact configuration

How can I remove the nan values from x?

ANSWER:

filter(lambda v: v==v, x)

works both for lists and numpy array
since v!=v only for NaN

ANSWER:

If you’re using numpy for your arrays, you can also use

x = x[numpy.logical_not(numpy.isnan(x))]

Equivalently

x = x[~numpy.isnan(x)]

[Thanks to chbrown for the added shorthand]

Explanation

The inner function, numpy.isnan returns a boolean/logical array which has the value True everywhere that x is not-a-number. As we want the opposite, we use the logical-not operator, ~ to get an array with Trues everywhere that x is a valid number.

Lastly we use this logical array to index into the original array x, to retrieve just the non-NaN values.

ANSWER:

For me the answer by @jmetz didn’t work, however using pandas isnull() did.

x = x[~pd.isnull(x)]

ANSWER:

Try this:

import math
print [value for value in x if not math.isnan(value)]

For more, read on List Comprehensions.