Re: [VOTE] CI tests RFC

On Fri, 2014-04-25 at 11:41 -0700, Stas Malyshev wrote: > Hi! > > > Would be nice if we could also create some policy regarding the usage of > > XFAILs (when should be a failing test marked as XFAIL, should we add the > > tests from the open bugreports as XFAIL by default, whose responsibility > > is to make sure that it will be fixed eventually, etc.). > > I agree. I didn't get into this but it definitely makes sense to have > some rules there. > > > I would also like to extend the current travis config a bit (we could > > have more exts, more axes for stuff like ts/nts builds, enable debug > > builds, so memory leaks are also triggering the test failures, etc.), > > You're more than welcome :) I've planned to get to some of it next - > i.e. going through the list of exts and see which ones we can support on > Travis - but any help would be great. > > > -- > Stanislav Malyshev, Software Architect > SugarCRM: http://www.sugarcrm.com/ > (408)454-6900 ext. 227 > Morning Stas, I got time to read through this morning properly, obviously, yes ... but I'm not sure about the options, maybe I missed some relevant conversation around it, not sure ... a few questions quick before I finish voting if you don't mind ... Update test seems like questionable action, but see that you and mike voted for it, can I hear reasoning for that ? I don't see when there would be a good reason to retain a broken commit for a week, isn't that going to just render all of this pointless if tests can fail for a week at a time ? Wouldn't it be better if all changes were done on branches, so they can be reviewed and integrated before being merged at all ? Maybe problematic for minor fixes but are they really the kind of thing that cause CI to bork ? Cheers Joe