SciPy

SciPy (website) is a popular tool set that includes NumPy, Pandas, and SymPy. SciKit Learn (website) is also supported. For evaluating exercises written using SciPy we're using the NumPy testing and unittest frameworks.

Hello Pi

Let's start with a Hello Pi exercise to explain the basics of evaluation in SciPy. Here is a solution file:

exercise.py

import numpy as np
print("Hello ", np.pi)

Evaluation of this simple "Hello Pi" exercise is also very simple. All we have to do is to create a class extending unittest's TestCase class and declare a test method inside. The test method will get the string printed in stdout and compare it with expected output ("Hello World").

evaluate.py

import sys
from unittest import TestCase
from numpy.testing import assert_equal
class Evaluate(TestCase):
def test_hello_pi(self):
import exercise # Imports and runs student's solution
output = sys.stdout.getvalue() # Returns output since this function started
assert_equal(output, 'Hello 3.141592653589793\n')

Let's see what we did in the above solution code.

  • 1st line, the sys module is imported which will help us get the output of student's code

  • 2nd line, TestCase class is imported to define our test class Evaluate and it is defined on the 4th line

  • 5th line, our first test function is defined

  • 6th line, the exercise module that is implemented by students is imported. Since the solution code is not in a function, it is enough to import the module to run the student's code. We have to import it inside the test function to be able to get the output.

  • 7th line, we get the output of exercise module. sys.stdout.getvalue() does exactly what we want here. It will only get the strings written to stdout since the test function has started. So you cannot get any outputs written before the test_hello_world() has begun.

  • 8th line asserts whether the output is correct and returns feedback if it's not.

Python 3

SciPy is based on Python and the documentation for Python covers a number of useful test scenarios. Please review those before continuing.

Example 1: NumPy

exercise.py

import numpy as np
a = np.array([1,2,3,4])
b = a.reshape(2,2)

evaluate.py

from unittest import TestCase
from numpy.testing import assert_equal
class Evaluate(TestCase):
def test_array(self):
import exercise
assert_equal(exercise.a, [1, 2, 3, 4])
assert_equal(exercise.b, [[1, 2], [3, 4]])

Example 2: Pandas

exercise.py

import pandas as pd
dates = pd.date_range('20130101', periods=6)

evaluate.py

import datetime
from unittest import TestCase
from numpy.testing import assert_equal
class Evaluate(TestCase):
def test_array(self):
import exercise
assert_equal(exercise.dates[0].date(), datetime.date(2013, 1, 1))
assert_equal(exercise.dates[-1].date(), datetime.date(2013, 1, 6))

Example 3: Data Loading From File

exercise.py

import pandas as pd
data = pd.read_csv("data.csv", header=None)

data.csv

3,2,1

evaluate.py

from unittest import TestCase
from numpy.testing import assert_equal
class Evaluate(TestCase):
def test_array(self):
import exercise
assert_equal(exercise.data.to_numpy().ravel(), [3,2,1])

Example 4: Data Loading From Url

exercise.py

import pandas as pd
data = pd.read_csv('https://storage.googleapis.com/applied-dl/heart.csv')

evaluate.py

from unittest import TestCase
class Evaluate(TestCase):
def test_array(self):
import exercise
self.assertTrue(exercise.data.to_numpy().size > 0)

Example 5: SymPy

exercise.py

from sympy.physics.units import mass, acceleration
from sympy.physics.units.systems.si import dimsys_SI
F = mass * acceleration
deps = dimsys_SI.get_dimensional_dependencies(F)

evaluate.py

from unittest import TestCase
from numpy.testing import assert_equal
class Evaluate(TestCase):
def test_array(self):
import exercise
assert_equal(exercise.deps, {'length': 1, 'mass': 1, 'time': -2})

Example 6: SciKit Learn

exercise.py

from sklearn import datasets
from sklearn import svm
digits = datasets.load_digits()
clf = svm.SVC(gamma=0.001, C=100.)
clf.fit(digits.data[:-1], digits.target[:-1])
results = clf.predict(digits.data[-1:])

evaluate.py

from unittest import TestCase
from numpy.testing import assert_array_equal
class Evaluate(TestCase):
def test_array(self):
import exercise
assert_array_equal(exercise.results, [8])