ABL Unit testing frameworks - Comparison

Posted by DivyaTheja on 27-Jun-2014 23:23

Here is a comparison of different Unit testing frameworks available in the market including ABLUnit which will be available soon to the customers. We will be glad to hear your experiences with these tools, please let us know your comments/suggestions.

 

ABL Unit

OE Unit

Pro Unit

File type support for developing tests

Supports class and procedure files as tests

Supports only class files as tests

Supports only procedure files as tests

Annotations

Required:

-        Before

-        Setup

-        Test

-        TearDown

-        Ignore

-        After

 

Required:

-        BeforeClass

-        Before

-        After

-        Test

-        Ignore

-        AfterClass

 

Doesn’t require annotations. But test case procedure should contain following internal procedures:

-        Initialize

-        setup

-        test<Name>

-        teardown

-        dispose

Developed in

Java(Tooling) and ABL(framework)

ABL

ABL

Error Handling

Yes

Yes

Yes

Data type Support

All ABL data types

All ABL data types

All ABL data types

Assertions provided

Equals

HasDeterminateExtent

IsAbstract

IsAvailable

IsDecimal

IsEmpty

IsFalse

IsFinal

IsIndeterminateArray

IsInt64

IsInteger

IsInterface

IsNegative

IsNull

IsPositive

isTrue

IsType

IsUnknown

IsZero

IsZeroOrNegative

IsZeroOrPositive

NonZero

NotAbstract

NotAvailable

NotEmpty

NotFalse

NotFinal

NotInterface

NotNull

NotNullOrEmpty

NotNullOrZero

NotTrue

NotType

NotUnknown

NotZero

AreEqual

AreNotEqual

AreNotSame

Fail

HandleAssert

IsFalse

IsNotNull

IsNull

IsTrue

ReplaceNull

assertTrue

assertFalse

assertNull

assertNotNull

assertEqualsChar

assertEqualsInt

assertEqualsHandle

assertEqualsDate

assertEqualsDecimal

assertEqualsLogical

assertEqualsTT

assertEqualsStrictTT

assertEqualsLongchar

assertEqualsInt64

assertEqualsDateTime

assertEqualsDateTimeTZ

assertEqualsMemptrContent

assertEqualsFile

assertQuicker

Testing Exceptions

Yes

Yes

Yes

Wizards for adding tests

Yes (Integrated with PDS OE)

No

No

Command Line Support

Yes

No

Yes

GUI

PDS OE

Can be embedded into PDSOE

Desktop App

Nightly Builds

Yes (Provided ANT task)

--

Can be used with PCT ANT Task

Results

Separate view is provided to show unit test results

-        Navigate to Next and Previous failures

-        Rerun tests

-        View history of last 10 results

-        View only failures

-        Go to particular test code

Pluggable view is provided to show unit test results

Summary in tree structure provided

 

Import results

Results can be imported by ABLUnit, JUnit, Jenkins.

No option available to import

No option available to import

Export results in different formats

No custom export options.

No export option

The results can be saved in different formats

-        XML with SLST

-        JUnit XML

-        CSV file

Plain text file with indentation

Export and Import results from UI

Can only import the results file

Cannot be exported/Imported

Results can be exported using Save As option.

All Replies

Posted by Rom Elwell on 17-Jul-2014 08:19

Do you have any additional details with regards to the release date?

Posted by andrew.may on 18-Jul-2014 03:52

Is there any chance that ABL Unit will be getting a mocking framework in the future?

We really need mocks/stubs to test interactions between objects, but generating these by hand in OOABL is very laborious & produces huge amounts of extra code that then has to be maintained.

This is the reason that we stopped using OEUnit in the past (even though we're very keen on unit testing).

Posted by Peter Judge on 18-Jul-2014 07:58

I don't think there are any formal plans to do so yet. Please add it to the Ideas section so that it can be evaluated/prioritised by Product Mgmt.  Ideas at https://community.progress.com/community_groups/products_enhancements/i/openedge/default.aspx
 
Do you have experience with mocking frameworks? Are there any or any whose approach you'd recommend?
 
-- peter
 
[collapse]
From: andrew.may [mailto:bounce-andrewmay@community.progress.com]
Sent: Friday, 18 July, 2014 04:54
To: TU.OE.Development@community.progress.com
Subject: RE: [Technical Users - OE Development] ABL Unit testing frameworks - Comparison
 
Reply by andrew.may

Is there any chance that ABL Unit will be getting a mocking framework in the future?

We really need mocks/stubs to test interactions between objects, but generating these by hand in OOABL is very laborious & produces huge amounts of extra code that then has to be maintained.

This is the reason that we stopped using OEUnit in the past (even though we're very keen on unit testing).

Stop receiving emails on this subject.

Flag this post as spam/abuse.

[/collapse]

Posted by andrew.may on 18-Jul-2014 08:56

I've only got experience of Java mocking.  For that, we're using Mockito as our main mocking framework & like it a lot.  

The Mockito API  makes for very nice test code & is popular enough to have been ported to a variety of different languages).

With it you can do things like: 

import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;

import org.junit.Before;
import org.junit.Test;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;


public class MyClassTest {

    @Mock private MyDependentClass mockMyDependentClass;
    private static String REQUIRED_PARAM_VALUE = "myParam";

    @Before
    public void setUp() throws Exception {

        // initialise mocking using annotations 
        MockitoAnnotations.initMocks(this);

        // Setup the mock to return a value if getNumberOfThings() is called with a parameter that .equals(REQUIRED_PARAM_VALUE)
        when(mockMyDependentClass.getNumberOfThings(eq(REQUIRED_PARAM_VALUE))).thenReturn(1);
    }

    @Test
    public void testThatdoStuffIsCalledWithMyParam() {
        // Create an instance of MyClass which uses the mock dependency
        MyClass classUnderTest = new MyClass(mockMyDependentClass);

        // Run the method that we're testing
        classUnderTest.doStuff();

        // Verify that the mock's method was called with a parameter that .equals(REQUIRED_PARAM_VALUE)
        verify(mockMyDependentClass).getNumberOfThings(eq(REQUIRED_PARAM_VALUE));
    }

}

... with MyClass.java being:

public class MyClass {

    private MyDependentClass dep;

    MyClass(MyDependentClass dep) {
        super();
        this.dep = dep;
    }

    public void doStuff() {
        // do some processing
        int numThings = dep.getNumberOfThings("myParam");
        // do some processing using numThings
    }
}

We also use Powermock to extend Mockito for some edge cases (like mocking statics) - that's more of a "nice to have".

Posted by andrew.may on 18-Jul-2014 09:08

Idea added - feel free to upvote :)

Posted by Stefan Drissen on 28-Nov-2014 08:28

I just thought I would have a quick play with this, and am only seeing the internal procedures - were functions really skipped?

Posted by DivyaTheja on 25-Feb-2015 10:45

Stefan,

ABLUnti testing Framework doesn't support functions as tests.

Posted by Lieven De Foor on 14-Dec-2015 03:13

As far as I know the current OEUnit can also be run from PCT. Could this be adjusted in the comparison?

Source: github.com/.../OEUnit

Posted by Mark Abbott on 25-Jan-2016 07:48

I wrote a mocking/testing doubles framework, to work alongside OEUnit, last year called OEMock. It's available on Github here: github.com/.../OEMock

So far, it works pretty well - I've been able to test it against a large production code set, and resolve issues from there. At the moment, it only support OEUnit, but could extend to ABLUnit with some changes. As ever, it's open source, so any issues logged or pull requests would be gratefully received.

Hope that helps,

Mark

Posted by Mark Abbott on 25-Jan-2016 07:49

I agree - I think that there are some features missing from the OEUnit side - for example Data Providers, and (currently undocumented) fixtures support. It can also produce test reports that can be read in to a CI engine in serveral different formats  (see the documentation for how to do this.)

This thread is closed