Read Ruby source - 2. Run test

Structure of test

Display all the make tasks to see the test tasks.

$ make help
                Makefile of Ruby

targets:
  all (default):   builds all of below
  miniruby:        builds only miniruby
  encs:            builds encodings
  exts:            builds extensions
  main:            builds encodings, extensions and ruby
  docs:            builds documents
  run:             runs test.rb by miniruby
  runruby:         runs test.rb by ruby you just built
  gdb:             runs test.rb by miniruby under gdb
  gdb-ruby:        runs test.rb by ruby under gdb
  check:           equals make test test-all
  test:            ruby core tests
  test-all:        all ruby tests [TESTS=<test files>]
  test-rubyspec:   run RubySpec test suite
  update-rubyspec: update local copy of RubySpec
  benchmark:       benchmark this ruby and COMPARE_RUBY.
  gcbench:         gc benchmark [GCBENCH_ITEM=<item_name>]
  gcbench-rdoc:    gc benchmark with GCBENCH_ITEM=rdoc
  install:         install all ruby distributions
  install-nodoc:   install without rdoc
  install-cross:   install cross compiling staff
  clean:           clean for tarball
  distclean:       clean for repository
  change:          make change log template
  golf:            for golfers

see DeveloperHowto for more detail:
  https://bugs.ruby-lang.org/projects/ruby/wiki/DeveloperHowto

$ make test : test for ruby core.
$ make test-all : test including ruby standard libraries.
$ make rubyspec : run RubySpec test suite

Run test

In this time, let's run test.

make test

$ view uncommon.mk
...
test: test-sample btest-ruby test-knownbug
...

$ make test
...
PASS all 1010 tests
...

make test-all

$ make test-all
...
/Users/jun.aruga/git/ruby/test/rubygems/test_bundled_ca.rb: cannot load such file -- openssl
Skipping `gem cert` tests.  openssl not found.
...

Mac's openssl which is installed by homebrew, was ignored.
https://github.com/ruby/ruby/blob/c8b3f1b470e343e7408ab5883f046b1056d94ccc/ext/openssl/extconf.rb#L62
http://stackoverflow.com/questions/17181075/openssl-error-installing-ruby-2-0-0-p195-on-mac-with-rbenv
Need to set path.

$ ./configure --prefix=/usr/local/ruby-2.3.0 --with-openssl-dir=`brew --prefix openssl`

$ make clean

$ make

$ make test

$ make test-all
[10139/15893] TestProcess#test_deadlock_by_signal_at_forking = 61.56 s
  1) Failure:
TestProcess#test_deadlock_by_signal_at_forking [/Users/jun.aruga/git/ruby/test/ruby/test_process.rb:2083]:
pid 97685 killed by SIGABRT (signal 6)
...
Finished tests in 483.736223s, 32.8443 tests/s, 4630.4161 assertions/s.
15888 tests, 2239900 assertions, 1 failures, 0 errors, 40 skips
...

Test has 1 failure. it can be possible as its ruby is develop version.

Do "make install" for later on rubyspec test.

$ make install

Rubyspec

Check spec directory. There are only 2 files.

$ ls -l spec/
total 16
-rw-r--r--  1 jun.aruga  staff  879 Dec  6 20:29 README
-rw-r--r--  1 jun.aruga  staff  803 Dec  6 20:29 default.mspec

Read README. I found I have to run "make update-rubyspec" to download spec files.

$ view spec/README

When I read the Makefile, update-rubyspec is only to do "git clone" rubyspec, mspec source.

$ view Makefile
...
update-mspec:
    @$(CHDIR) $(srcdir); \
    if [ -d spec/mspec ]; then \
      cd spec/mspec; \
      echo updating mspec ...; \
      exec git pull; \
    else \
      echo retrieving mspec ...; \
      exec git clone $(MSPEC_GIT_URL) spec/mspec; \
    fi

update-rubyspec: update-mspec
    @$(CHDIR) $(srcdir); \
    if [ -d spec/rubyspec ]; then \
      cd spec/rubyspec; \
      echo updating rubyspec ...; \
      exec git pull; \
    else \
      echo retrieving rubyspec ...; \
      exec git clone $(RUBYSPEC_GIT_URL) spec/rubyspec; \
    fi

...

Target files were downloaded.

$ ls spec/
README         default.mspec  mspec/         rubyspec/

$ vi uncommon.mk
...
test-rubyspec: test-rubyspec-precheck $(arch)-fake.rb
    $(RUNRUBY) -r./$(arch)-fake $(srcdir)/spec/mspec/bin/mspec run -B $(srcdir)/spec/default.mspec $(MSPECOPT)
...

$ make test-rubyspec
...
1)
Date#<< raises an error on non numeric parameters FAILED
Expected TypeError
but got NoMethodError (undefined method `-@' for :hello:Symbol)
/Users/jun.aruga/git/ruby/spec/rubyspec/library/date/minus_month_spec.rb:27:in `block (3 levels) in <top (required)>'
/Users/jun.aruga/git/ruby/spec/rubyspec/library/date/minus_month_spec.rb:4:in `<top (required)>'
[/ | ==================100%================== | 00:00:00]      1F      0E

Finished in 52.027629 seconds

3487 files, 24914 examples, 175758 expectations, 1 failure, 0 errors, 0 tagged
...

1 failure.

Because it is develop version..

Ruby CI

Test status for each platform.
http://rubyci.org/

Jun's Blog

Output, activities, memo and etc.