Code Temple

ReflectionException: Method PHPUnit_Framework_Warning::Warning() does not exist

2011-12-05T14:50:00.001-08:00

If you ever got error like this:

ReflectionException: Method PHPUnit_Framework_Warning::Warning() does not exist

It means you have no tests defined in your unit test file.

Thoughts about: PHP - CQRS - Axis2 - Java - Quercus - ES - DDD - Messaging

2011-05-10T15:08:00.001-07:00

Year ago I have started to work on a small PHP framework that had a purpose to help me out to implement CQRS based architecture for a pretty large e-commerce system. It took me 4-5 month but I did create first version and went live. Next month it will be one year when CQRS based system implemented only with PHP is live and rolling. And all I can tell you that I was never happier than now.
We are able to manage complexity, we are able to be agile from business perspective, we are able to maintain and develop new features while being a pretty small team. It's awesome, but still now I have started to think about how I could improve all this even more.

So let's see what we got now and what bothers me.
At very top level our system is based on SOA principle, we are organizing ourselves around services. One specific thing is those services are domain focused services. What we did at the beginning was that we have identified domain models and then we did draw boundaries which actually has formed bounded contexts (Domain Driven Design). For us bounded context is like a sub-system, some specific domain focused system, with a well-defined interface. For example "Sales", "Support" sub-systems/bounded contexts. The most important thing here is - well-defined interface. Because as you understand once you do got an interface, behind it you can have anything you want.

So we got our BCs. Now inside it we got CQRS based system. Well not all of them got it, but most of them. We got read side with simple "Query" classes designed for our needs. It's simple php classes doing queries to our reporting storage (for some we use mongo, for some MySQL). Then we got our write side. If we would be going from top-to-bottom first of all we got our Application layer. This layer got application services which basically publishes interface what you can do with our domain. This layer got no logic apart how to publish message to some messaging system. So once message is in a bus (we are using RabbitMQ and we are more than happy) on the other end we got X PHP daemons running and consuming those messages. Those daemons are getting messages from messaging system, then it builds a command and executes it. Execution means that we are performing some action on our domain model which again is written in PHP. Our domain processes the request and generates events that represents state changes, those events then are stored to event store (we are using mongo to store events). Lastly event handlers are notified that some events has occurred and thats where we got our reporting updated with data.
We also got event versioning so that our domain model would support only the latest events.

And all this works perfectly. Everything is decoupled, has its place and simply its very clear what is where, why, how and when.
Gods knows how many times I was saved because of events I got. Generate new reports, analyze events and see why the hell one or another thing happened. Because of CQRS it was easy to split work load and to simplify domain model like N times ? One model - one purpose read or write, simply as that.

But now I want to improve all this and was thinking about bit different setup. Concepts, patterns remains as they were it's just technologies that I want to replace.

First of all - web services. PHP was never meant to be used to create web services. It works, it will do the job for you. I have done that and it is running and partners are using, but it's "weak". I mean what about WS-* ? I found only one project that is trying to implement these specifications WSO/Oxygen Tank. But when I did try it, it was hard to use and simple not working. Also for all such tasks you need tools to make your life easier while developing, deploying. So I thought this is where I will draw a line and I will try to replace it with something else. And my first candidate was Axis2 Java implementation. We got WS-*, we can create and deploy web services very easily and fast and features like "hot deployment" really helps you a lot. Also it's proven and tested technology.
So then I thought what if as SOAP engine I would use Axis2 and application layer would be implemented in Java as well. It is a thin layer with no logic just simple messages publishing mechanism. And then at the other end I would have everything as it is now - PHP domain implementation.
In addition to that I have added quercus to this stack. Its PHP implemented in Java. And this allows me to create a front-end with a language that is most suited for it. And because it is implemented in Java I am getting all Java abilities.

One last thing. To answer a question why PHP ? It's simple as that - its easy to find developers, PHP developers are cheaper, simply why not ? Tell me one good reason why startup company shouldn't be using PHP as back-end language ? An besides if you will need to replace it with Java, Scala, .NET or with anything else - who will stop you ? It's all about "who" will be consuming messages, no?

So, this is what I am thinking at the moment. Any comments are more than welcome ...

DDD - too large Agregate root

2010-07-17T15:38:00.000-07:00

Last week, while I was working on some code for my current project at work I ran into problem where my aggregate root had too many child entities, way too many. My aggregate root snapshot was about 20 MB and that of course gave me a sign that it is wrong.

Model was rather simple one. I had to create a model for voucher system. Every deal that our company creates would need about 100k vouchers. Every deal has vouchers that provides different so called gifts to customers. And those deals are created pretty often, so we are talking about millions of vouchers.

Now voucher was identified as entity because it has behaviors. And as an aggregate for them I have presented Stock entity. It hold references to all vouchers and those were accessible through it. Also it was responsible for generating vouchers and exporting them. I was using a rule that there always should be a higher 'thing' for my entities and that thing should know what entities exists out there. It's like for licenses we have stock, for invoices ledger and so on.
Now why it is wrong ? (Well I think it is wrong)

First of all in this way I am transferring repository responsibility to other object - stock, ledger etc. I am creating repository that holds all data in-memory while I do not need it. What I really need is just another domain service.
So, let me explain how I have re-modeled my domain here.

Stock aggregate root remains but voucher becomes an aggregate root. I do present new aggregate root - VoucherGenerator that has knowledge about voucher code generation strategies. And what I do to solve problem is I add new services like GenerateVouchersService, VoucherLookupService. Instead of having behavior on stock 'GenerateVouchers' i move this to Generator AR and then use this in service GenerateVouchersService to actually create new vouchers and add those to repository.
In this was my repository does what it suppose to do - acts as in-memory domain objects collection and my domain entity becomes free of millions child entities. Of course it still has a map of voucher code - UUID in my stock aggregate and I use it to get voucher UUID that later is used to get entity from repository.

Of course Stock AR still will have lot of data in it, but what we can do is introduce 'ClearOldVouchers' behavior. Or remove voucher once it was used. And if at some point later I will have to restore my entities, because of event sourcing I am still able to do that.

So as you see, in this way we have solved large-aggregate-root problem and at the moment it seems to be the best solution for me.

Any comments ? Have you any other solution ?

Applying Scrum - Sprint Planning - Mind Map

2010-03-06T14:43:00.000-08:00

Last Thursday our team have finished 3rd sprint. Despite some issues, we have achieved our goal and on Monday we will have our demo.

This evening I was sitting here and was analyzing how we were progressing during sprint. One new thing that we have tried for this sprint was - 'no story splitting into tasks' thing. What we did was, that we just created few tasks for the first day of the sprint and agreed to create the other tasks on the fly, whenever we will need it during sprint. After that we just brainstormed about all sprint stories and what solutions we can suggest for product owner.

So now when I am looking back to how it worked, I really like it. First of all sprint planing meeting was not so 'hard'. We were just sitting and discussing things. And if I try to compare it with sprint planning that happened before this one, so I could say that last one was much more productive. Because we were not forced to commit ourselves on what exactly we will have to do and how long it will take, like 'create x files' and 'create y things' and so on. But instead we analyzed in general how we will do it and what the best solution we can offer to our product owner. I think we have covered more things in this way compared to the splitting-into-concrete-tasks way.

Now during sprint, in every daily scrum we have defined new required tasks and agreed who will work on each task. What I liked the most about all this was that it was so easy to create those tasks. It was very clear what we need to do. Every morning there were new challenges that we had to take if we wanted to achieve our goals. And now when I have putted all those sprint tasks on the wall and joined them, I noticed that I have got a web:

Story A -> task 1 -> task 2
task 1 -> task 3
....

And that gave me an idea.

This sprint we will do the same thing, just I will try one additional activity. I will try in next sprint planning meeting to use a mind map tool. While we will brainstorm about each story, I will time-box each session for 10-15 minutes and will see with what trees we will come up. Then I will print all those trees and during daily scrum we will use those to define tasks.
What I hope to achieve is - better solution. Because during brainstorm session, of course we will define some solution for a story, but first solution, at the most of times, is the one that will be re-factored. So I hope these mind-maps will have those first solutions and during sprint, in daily scrums, we will see what solutions we already have and perhaps we will define some new ones and better ones and in this way we will improve quality of our software.

Let's see if it works.

Convert problems to challenges

2010-03-02T13:24:00.001-08:00

We had sprint planning meeting and it was really hard meeting. Lot of unknown things about stories and problems were there and there and it seemed that all this product is one big problem. So what our product owner suggested, was like this (credit goes to Charlotte Way):

Let's change the word problem to challenge and when you have a problem, instead of saying I have a problem, say I have a challenge.

And you know it really works. It was not so depressive anymore to talk about those problems, because there were no problems! Product now is a challenge and bad 3rd party API is a challenge, that bug is a challenge and when you get used to that word it really works, and everything looks much easier.

So from that day, I do not have any more problems, there are only challenges!

Applying Scrum - Part 1

2010-03-02T12:11:00.000-08:00

Few weeks ago I read a book, which basically was about how practically scrum was applied in one big company. Good thing about that book was that it contained only practical examples how Scrum worked for them, no theory, only pure practical examples.
That book helped me a lot, so I thought perhaps it would be good idea to share how we do scrum and perhaps someone will find it useful.

It's about team members commitment
Challenge - make team feel responsible for a product

At the beginning, when I have started to implement Scrum, everything was pretty ok. We have implemented all those basic scrum rules like daily scrum, we had backlog, product owner, scrum master, agreed on sprint length and so on. But despite of lot of other challenges, I will cover those later, after few sprints we were failing to deliver what we have promised and deliverables was note actually DONE.
So I was thinking were is that challenge? why this team is failing ?
When I have started to analyze, what actually we were doing during sprint, I found that team was not working as a team, but they rather just waited till what I will say how and what we should do! Daily scrum was like "so what should I do now ? I did that, and that, what next ?" And when you fall into this, when you start thinking what all should do, you are starting actually to manage them and that is wrong very wrong.
Team must feel responsible for what they are delivering. You have a goal and you are responsible to do everything what you can to achieve that goal, there is always something you can do, you just have to feel that this is your product and you will see that there is a thousands of tasks that you can start working on. And there should be no "you" and "me", there must be a team and everyone must understand that if I will fail to do something - all team will fail.
And what happens when there is one person who is creating and assigning tasks and he is the only one who sees the whole picture? You always can blame him and only that person will be responsible.

So, lesson that I have learned is:

Before you start sprinting, make sure that team understands what that word "team" really means. Try to make them feel that they are a part of something that they are about to create. And also explain people what each of team member is and should be expecting from each other - that would be responsibility.

I think that if you want to be a very very productive team, first of all, all team members must feel responsible for the product and treat it like it would be their own baby.

DDD/CQRS/Event sourcing and Agile/Scrum - Is it so obvious?

2010-02-28T14:48:00.000-08:00

I was sitting here and was creating a roadmap for a system that I am currently working on. Nothing special, but still, it's big eShop system with big backend system and has lot of integrations with 3rd party systems and has complex business rules and so on.
Anyway, now why I am writing this, is because I think that all who are reading about CQRS, Event sourcing, Agile, Scrum etc., should be interested in it. Perhaps all top guys like Greg Young, Udi Dahan and lot of others knows that, but I have tried to ask google and got zero results on what I am about to write.
Or perhaps it's so obvious that no one is writing about it? let's see

I will not go into details what is Scrum and Agile and CQRS and Event sourcing, I assume that you know what it is, and instead of that I will share my thoughts on how all these 'things' works together and what a great value it can bring to you.

So, first of all some preconditions:

My team works in Scrum environment (we pass NOKIA test)
We have applied CQRS architecture along with Event sourcing
We use DDD

Now, as I was saying, because I am ScrumMaster, so I was helping our Product Owner to create a roadmap for our current project. We have a loads of work/features to implement and also there is a very big pressure from stakeholders to go live as soon as possible. There is along story why we need go live as soon as possible, but thing is that we must, otherwise business will suffer a lot. So puzzle is something like this:
- You have to deliver X features till Y date, but to develop those X features with quality there is no way you can do it till Y date. So what to do ?

So I was splitting stories and batching them and preparing them for Product Owner that she could prioritize those and then I have realized:

What you gain from CQRS? You have write only domain (there are of course more, but this is the most important for this article)
What you gain from event sourcing ? You have a full audit of your system. You know every single step that was made on your system, I mean every.
How product is build in Agile environment? You do it step by step, creating tested and shippable increments that at the end of the day builds up a product

So, I think some of you are getting the picture already. Now if we have a really good user stories, our stories are written in INVEST way, so we can create the most important related features/stories in first sprints and if that is enough for business - we can go live. Yes, that is right, go live. Deploy system to production environment, start promoting and do all the rest that business needs.
Meanwhile Scrum team will start implementing future features, that are related to our deployed features and requires generated data from them. And thing is that once we are done with our so called future features, we can take out event log from our production environment, plug it in to our staging environment, click 'replay' button and wait till our data, generated in production environment, will be grabbed by newly implemented features. Those features then will update their state (internal event handlers), update required storages (external event handlers) etc. And once it done, we can go live again, but this time our system will have newly implemented features, and moreover, it will look like those features were there from the beginning!
For example we need reporting functionality for sales department (we call it bounded context, because its separate, autonomous mini application) that says how many sales were done and so on. So once we have reporting features implemented we can replay events (sales events) that happened on production environment and let 'reporting features' do what they suppose to do when that kind of event occurs! Isn't it amazing ? You can create future while present is running, gathering data and then integrate all this and put it back for production usage, like it was there from the beginning! (I am repeating myself i guess :))

Of course, I hear you saying that there are features that can not be plugged in like that, and I totally agree with you, but perhaps it means then that those must go together in one sprint, or must be rethought if those, related features, are really important to business and must be included in sprint. Because at the end of the day, its all about business and what value one or other feature will create.

So, that would be it. I hope it will help someone. I really would like to get some feedback on this, so if you have something to say about this, you are more then welcome.

Zend Framework poster

2009-08-24T20:35:00.000-07:00

Few weeks ago my friend showed me a very nice ZF poster that Germany company, called "Mayflower" is spreading around the world for free, just because they likes Zend Framework.
So, i also asked for one, and here it is:

If you want one, just write a short email with asking for one to bjoern.schotte@mayflower.de and in few weeks you will have your poster.

It is very nice to see that there are such companies as "Mayflower", which are not just using this wonderful framework, but also makes such things to inform people that there is such framework and that you definitely should have a look in it.

PHP objects serialization

2009-01-09T06:54:00.000-08:00

This week I was working on developing two systems integration mechanism. Integration was based on XmlRpc web services. After a successful web services framework development I have started to implement businesses logic and stuck on objects serialization.

When I tested serialization of simple objects, that had only few methods and no properties, everything was working just fine. I serialized object on one system, send to remote one and unserialized for usage. But when I have added properties to object, the remote php web service was not able to unserialize received object. It took me some time to find out that those private properties and was the real problem. Because of unknown reason private and protected properties are not handled nicely in serialization process private variables are prefixed with class name + #65533; and protected with #65533;. Anyway the fix was to encode serialized string to base64 and on remote side before unserialization decode from base64.
For example:

...

class LeetObject
{
private $property;
protected $_property;

public function test()
{

}
}

$obj_category = new LeetObject();
$b64_encoded_and_serialized = base64_encode(serialize($obj_category));

// Send somewhere
// And on remote system

$obj_category = unserialize(base64_decode($obj_category));

// Use object

...

Hope this will help someone.