Well, it depends. There was a recent patch that changed the behaviour for skipping tests. (a required test was run regardless of conditions, and failed in execution not error out at load time)
So, I need to see your local.conf. In short, if a test
is added in TEST_SUITES it will now error out, so if you have a sato image
with TEST_SUITES = "ping ssh rpm smart" those are considered required tests and none of them are skipped (as a module, individual classes can be skipped as a dependency). 
It works that way because a test usually does some kind of a check: e.g packagement-manament is in IMAGE_FEATURES but we also know for sure that a certain image (e.g core-image-sato) _must_ have packagement-management in features so we error out assuming that something went wrong and 
that's not the right image.
Oh, btw, rpm and smart are in the defaults for sato (so if this is an ipk/deb build, then you need to overwrite TEST_SUITES completely).
Alternatively you could use TEST_SUITES = "ping ssh auto", that will indeed skip non-applicable tests.

One more thing, here it is how it used to work before http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2dfc0e1cc72491d24536792d4bc1d5f978634671

core-image-sato, TEST_SUITES = "ping ssh gcc" 

NOTE: Test modules  ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
NOTE: Found 5 tests
test_ping (oeqa.runtime.ping.PingTest) ... ok
test_ssh (oeqa.runtime.ssh.SshTest) ... ok
WARNING: Test gcc is required, not skipping
test_gcc_compile (oeqa.runtime.gcc.GccCompileTest) ... FAIL
test_gpp_compile (oeqa.runtime.gcc.GccCompileTest) ... FAIL
test_make (oeqa.runtime.gcc.GccCompileTest) ... FAIL

======================================================================
FAIL: test_gcc_compile (oeqa.runtime.gcc.GccCompileTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/stefans/yocto/poky-work/meta/lib/oeqa/runtime/gcc.py", line 20, in test_gcc_compile
    self.assertEqual(status, 0, msg="gcc compile failed, output: %s" % output)
AssertionError: gcc compile failed, output: sh: gcc: command not found

======================================================================
FAIL: test_gpp_compile (oeqa.runtime.gcc.GccCompileTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/stefans/yocto/poky-work/meta/lib/oeqa/runtime/gcc.py", line 26, in test_gpp_compile
    self.assertEqual(status, 0, msg="g++ compile failed, output: %s" % output)
AssertionError: g++ compile failed, output: sh: g++: command not found

======================================================================
FAIL: test_make (oeqa.runtime.gcc.GccCompileTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/stefans/yocto/poky-work/meta/lib/oeqa/runtime/gcc.py", line 32, in test_make
    self.assertEqual(status, 0, msg="running make failed, output %s" % output)
AssertionError: running make failed, output sh: make: command not found

----------------------------------------------------------------------
Ran 5 tests in 11.167s

FAILED (failures=3)

--

This is how it works now:
--
NOTE: Test modules  ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
NOTE: Found 5 tests
test_ping (oeqa.runtime.ping.PingTest) ... ok
test_ssh (oeqa.runtime.ssh.SshTest) ... ok
ERROR

======================================================================
ERROR: setUpModule (oeqa.runtime.gcc)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/stefans/z/poky/meta/lib/oeqa/runtime/gcc.py", line 8, in setUpModule
    skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
  File "/home/stefans/z/poky/meta/lib/oeqa/oetest.py", line 108, in skipModule
    "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
Exception: 
Test gcc wants to be skipped.
Reason is: Image doesn't have tools-sdk in IMAGE_FEATURES
Test was required in TEST_SUITES, so either the condition for skipping is wrong
or the image really doesn't have the requred feature/package when it should.

----------------------------------------------------------------------
Ran 2 tests in 6.255s

FAILED (errors=1)
--


Regardless of the change, here it how it looks with TEST_SUITES = "ping ssh auto":
skipped "gcc: Image doesn't have tools-sdk in IMAGE_FEATURES"
The disadvantage of auto is that we don't control the order tests run.
So in the particular case of rpm/smart most likely it has to do with the fact that those are in the default suite:
- from meta/classes/testimage.bbclass:

DEFAULT_TEST_SUITES = "ping auto"
DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp vnc date perl ldd gcc rpm smart dmesg"

TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"

So bottom line: "rpm smart" shouldn't be added in TEST_SUITES for ipk/deb images as they are considered mandatory tests and don't get skipped.