On Fri, May 27, 2011 at 12:42 PM, Frank Shearar <frank.shearar@gmail.com> wrote: > You miss the point of mocks. Mocks are there to allow you to test your > code quickly and cheaply against things that may not work in a quick > manner. Not likely. Hernán has been around the block and he speaks from long experience and deep thinking on software design. If you read his post a little more carefully, you'll see that he distinguishes between test doubles in general and mocks as a particular form of test double. He uses test doubles - you don't get 23,000 tests to run in 7 minutes without them. It's just *mocks* that are mostly unnecessary. Personally, I find that mocks are *very* useful in a few very specific situations, but they can cause problems it other situations. They're like a very specialized tool - invaluable when you need it, but kept in the bottom of the toolbox and not used very often. Where I find them useful is in testing communication. For example, I'll use a MockWritestream to throw an exception if the code under test writes something unexpected into the stream. It's handy when testing code that writes out files in a particular format, or communicates with remote systems over the network. It might also be useful to test communications across formal module boundaries, where it's particularly important that a specific protocol be used - eg, for something like Prevalayer. My favourite testing pattern is TrivialImplementation. Instead of using a test double or an expensive "real" object, I often create a cheap/trivial implementation of the objects that collaborate with the objects I'm testing. For example, Monticello2 has several repository classes. There are implementations that store code in a single file, in a directory of files, on a remote machine via sockets, on a web server via HTTP etc. There's also a trivial implementation that just uses an in-memory Dictionary for storage. It's a complete implementation of the repository protocol, and Monticello can use it "for real", but it's not as robust as the other kinds of repository. All the repository implementations, including MemoryRepository, have a suite of tests that ensure that they work correctly. But when I'm testing other parts of Monticello that interact with a repository, those tests use a MemoryRepository. I know that BDD folks like to talk about testing behaviour rather than state, but I don't find the distinction useful. Testing state breaks the encapsulation of the objects under test, and couples the test too closely to their internal implementation. Testing behaviour using mocks *does the same thing*; it just restricts the implementation in a different way. I find it's better to give the implementation a degree of freedom by testing "results". Figuring out what result you're looking for can be difficult - it requires thinking about what a passing test really tells you. Here's an example - let's say we we're testing an implementation of Set. Here's the version that tests state: | set | set := Set new. set add: 3. self assert: (set instVarNamed: 'array') = #(nil nil nil 3 nil) Here we test behaviour: | set | mock := MockArray new: 5 mock expect: (Message selector: #at:put: arguments: #(4 3)). set := Set withArray: mock. set add: 3. I prefer this approach. It gives the implementation a lot of freedom, while ensuring that it does what we really want: | set | set := Set new. set add: 3. self assert: (set includes: 3). In short, I agree with Hernán. Mocks can be useful, but they're often overused. In most cases, they're unnecessary. Colin

Hey! Stumbled into this thread but just HAVE to respond... Snipping heavily here: On 05/27/2011 11:11 PM, Colin Putney wrote: > without them. It's just *mocks* that are mostly unnecessary. Amen! I have been thinking the rest of the world had gone mad... > Personally, I find that mocks are *very* useful in a few very specific > situations, but they can cause problems it other situations. They're > like a very specialized tool - invaluable when you need it, but kept > in the bottom of the toolbox and not used very often. Right. [SNIP of good stuff] > I know that BDD folks like to talk about testing behaviour rather than > state, but I don't find the distinction useful. Testing state breaks > the encapsulation of the objects under test, and couples the test too > closely to their internal implementation. Testing behaviour using > mocks *does the same thing*; it just restricts the implementation in a > different way. So refreshingly to see someone that *also* notes that testing using mocks actually break encapsulation. Thank you, thank you... regards, Göran

2011/5/31 Göran Krampe <goran@krampe.se>: > Hey! > > Stumbled into this thread but just HAVE to respond... Snipping heavily here: > > On 05/27/2011 11:11 PM, Colin Putney wrote: >> >> without them. It's just *mocks* that are mostly unnecessary. > > Amen! I have been thinking the rest of the world had gone mad... > >> Personally, I find that mocks are *very* useful in a few very specific >> situations, but they can cause problems it other situations. They're >> like a very specialized tool - invaluable when you need it, but kept >> in the bottom of the toolbox and not used very often. > > Right. > > [SNIP of good stuff] >> >> I know that BDD folks like to talk about testing behaviour rather than >> state, but I don't find the distinction useful. Testing state breaks >> the encapsulation of the objects under test, and couples the test too >> closely to their internal implementation. Testing behaviour using >> mocks *does the same thing*; it just restricts the implementation in a >> different way. > > So refreshingly to see someone that *also* notes that testing using mocks > actually break encapsulation. Thank you, thank you... Or to rephrase, "documenting the precise nature of the colloboration between an object and its context documents that precise nature." In Colin's example, Set's collaboration with Array is purely an implementation detail and, as he notes, testing the implementation - whether checking that Set stores the value in the array, or what messages Set might send to Array - hurts you. I fail to see how that makes mocks bad. Unless you're railing against hype? In which case, I agree: there ain't no such thing as a free lunch, there is no silver bullet, etc etc. frank

First of all, I'm sorry for my English. (And I'll appreciate it very much if someone will point to mistakes of any kind in the following text.) I attempted to analyze and structure comments, questions and ideas presented in the thread and I think I found one thing that can be a starting point for the discussion (despite the fact the discussion already started… away back). I've got a strong feeling (I've got it several years ago and had a number of evidences, including several in this thread) that there are (at least) two very different (if not opposite) points of view on mock objects… and it seems to me, the root of this divergence is even deeper and more general — understanding of TDD. So, I think I have to explain my point of view. (Telling "my" here I claim to be an author of all errors, misinterpretations and nonsenses I'll present. But all the good, reasonable and interesting things were invented by other (clever) people.) So, why and how did I dare to claim TDD is not really popular among Smalltalkers? Because I still believe tests can drive (nearly) all stages of development: not just coding, but also design and even analysis (at least to clarify and formalize results of analysis). But I rarely (if ever) see any artifacts (tests of the kind). Perhaps I'm wrong, but most of the tests I saw were more "acceptance" tests, perhaps even written afterwards. Once again: very likely I'm wrong (and I'll be glad to be, actually). Why is it important for me to use TDD? Because, as a bad programmer, I need a help. I have a lot of things I must think about during development (won't enumerate, we all know them), so I want a tool that helps me not to think too much — at least, at the points where it is possible at all. (If I misinterpret the word "driving" here, please let me know.) Thus, if I write a line of code that is not directly(!) driven by a test, I don't TDD. If I make some up-front design I'm not actually test-driven. Even, if I simply go few steps designing top-down without tests at a very small single stage of development, TDD is broken. If you think it is too extreme and strict — ok, let's talk about "seamless TDD" vs., say, "partial TDD". (Yes, I know about steps of variable size during TDD etc. But I still think it's a bit another thing. Let's judge a bit later…) So, considering an example by Colin Putney: | set | set := Set new. set add: 3. self assert: (set includes: 3). vs | set | mock := MockArray new: 5 mock expect: (Message selector: #at:put: arguments: #(4 3)). set := Set withArray: mock. set add: 3. Isn't it obvious mocking array here is an overkill and the first test is better? For sure it is. But why? Because there's actually no development in the 1st test. Otherwise, what is it for: #add: or #includes:? Which feature this test makes me to add to the system-under-test (the Set)? I must select, I must think, thus I'm not driven by the test. Consider also (slightly) another issue: here we have a ready-to-use inner collection for the Set. But if I have no yet? Isn't it a common and very frequent situation during development: I have to implement a (sub)system that comprises several objects working together? (I think I don't have to provide examples here to prove.) Which one should I implement first? I don't know, I have to explore. I must analyze the system, i.e. divide it into subcomponents, find the ones independent and start by implementing them first. But what is this process, I've just described? It's a top-down up-front design. Is it test-driven? Obviously, no. Do I have an alternative? Yes, I can use mocks. With Mocketry I could write: SetTests >> testStoresElementsInInnerCollection [:collection | set := Set withArray: collection. [set add: #element] should strictly satisfy: [collection add: #element]] runScenario Of course, this is a very early step of development, as I'll have to implement an efficient algorithm to check, if element exists in the set before adding it… And with mocks it is very likely I'll be driven to implement it as a pluggable object. Probably, this can make Set a bit more flexible, doesn't it? On the other hand, it can introduce unnecessary complexity. Apparently, this is a good example to explore it further… While I must admit technique of mock object is not a silver bullet, and must be used carefully… just as any other tool. So… Conclusion 1: Yes, mock objects is a specific tool for specific situations. But as for me, I meet these specific situation almost as frequently, as other, non-specific(?) circumstances. Conclusion 2: I think mocks should be used, first of all, to discover different aspects of functionality and distribute it among various objects. (Documentation, optimization, etc. goes later… if goes at all…) When I start to work on a new feature, I first try to write an acceptance test. But I never can implement it 'as is'. And I hate to leave it red or mark it as ignored (in any way). So I try to transform it into something I can implement easily and fast. And often this leads me to a test with mocks. This make me to divide complex functionality into smaller blocks and factor them out to collaborators. The trick is to make those objects to behave (one-by-one and altogether)… But that's another (and big) topic. -- Dennis Schetinin

On 05/31/2011 04:59 PM, Frank Shearar wrote: > 2011/5/31 Göran Krampe<goran@krampe.se>: >> So refreshingly to see someone that *also* notes that testing using mocks >> actually break encapsulation. Thank you, thank you... > > Or to rephrase, "documenting the precise nature of the colloboration > between an object and its context documents that precise nature." > > In Colin's example, Set's collaboration with Array is purely an > implementation detail and, as he notes, testing the implementation - > whether checking that Set stores the value in the array, or what > messages Set might send to Array - hurts you. > > I fail to see how that makes mocks bad. Unless you're railing against > hype? In which case, I agree: there ain't no such thing as a free > lunch, there is no silver bullet, etc etc. Well, yes, railing against hype. But also railing against writing tests that work from "the inside" by using mocks. It is of course a matter of taste, style and in the end weighing pros and cons. But I have seen too much use of mocks that end up making the tests purely mirror the implementation - so even the slightest internal modification would break the test. regards, Göran

2011/5/31 Göran Krampe <goran@krampe.se>: > On 05/31/2011 04:59 PM, Frank Shearar wrote: >> >> 2011/5/31 Göran Krampe<goran@krampe.se>: >>> >>> So refreshingly to see someone that *also* notes that testing using mocks >>> actually break encapsulation. Thank you, thank you... >> >> Or to rephrase, "documenting the precise nature of the colloboration >> between an object and its context documents that precise nature." >> >> In Colin's example, Set's collaboration with Array is purely an >> implementation detail and, as he notes, testing the implementation - >> whether checking that Set stores the value in the array, or what >> messages Set might send to Array - hurts you. >> >> I fail to see how that makes mocks bad. Unless you're railing against >> hype? In which case, I agree: there ain't no such thing as a free >> lunch, there is no silver bullet, etc etc. > > Well, yes, railing against hype. But also railing against writing tests that > work from "the inside" by using mocks. It is of course a matter of taste, > style and in the end weighing pros and cons. But I have seen too much use of > mocks that end up making the tests purely mirror the implementation - so > even the slightest internal modification would break the test. PragProg were kind enough to publish a very well-timed article: http://pragprog.com/magazines/2011-06/practical-mock-advice The punchline? "Generally, there is one overarching reason for why you would use a mock. That reason is this: You care about an implementation detail in your code. And when do you care? You care when it drives behavior." frank