Design Judging after Dynamics

[Dunk Mckay]

I've started to think about what would happen if design judging was right at the end of a competition, after all the dynamic events.

Ok, I'm not actually suggesting that this be in any way implemented at competition, I'd feel very sorry for any team whose car was involved in a "racing incident". More from a purely hypothetical point of view, how would teams change their design approach if they thought about it this way?

A team with a good well thought out design methodology shouldn't need to change anything. But a team that takes risks, gambles on design decisions and assumes it can blag it's way through design judging would have to do some serious rethinking. If you turn up with underengineered components hanging off your car, or having failed to complete endurance due to predictable componenet failure, you're going to have a hard job justifying why you made it that way.

At the moment all we have to present takes us from concepts, through theory and then physical testing. But numbers can be fudged(intentionally or not), justifications for poor decisions made up, etc. I mean no disrespect to the judges here, they do a great job weeding out the good engineering from the bad, but we're all human (well most of us http://fsae.com/groupee_common/emoticons/icon_wink.gif) especially within the time constraints.

If everyone designed like this I suspect you'd have a lot more teams finishing endurance. Most failures would be down to misfortune rather than negligeance and poorly engineered parts/systems/teams. But among the larger number that did finish you'd have a wider, perhaps more even, spread of results. Ranging from teams that realize that they maybe don't yet have the means, the experience or the development to build a "top ten" car so settle for reliability first and foremost, and the teams that usually do have "top ten" cars, and generally only fail if they push things a little too far (like not enough cooling, or undersized flextures on wishbones).

Again this is just a hypothetical suggestion, and not something I think would actually work at competition.

EDIT-Grammar

[Dunk Mckay]

I've started to think about what would happen if design judging was right at the end of a competition, after all the dynamic events.

Ok, I'm not actually suggesting that this be in any way implemented at competition, I'd feel very sorry for any team whose car was involved in a "racing incident". More from a purely hypothetical point of view, how would teams change their design approach if they thought about it this way?

A team with a good well thought out design methodology shouldn't need to change anything. But a team that takes risks, gambles on design decisions and assumes it can blag it's way through design judging would have to do some serious rethinking. If you turn up with underengineered components hanging off your car, or having failed to complete endurance due to predictable componenet failure, you're going to have a hard job justifying why you made it that way.

At the moment all we have to present takes us from concepts, through theory and then physical testing. But numbers can be fudged(intentionally or not), justifications for poor decisions made up, etc. I mean no disrespect to the judges here, they do a great job weeding out the good engineering from the bad, but we're all human (well most of us http://fsae.com/groupee_common/emoticons/icon_wink.gif) especially within the time constraints.

If everyone designed like this I suspect you'd have a lot more teams finishing endurance. Most failures would be down to misfortune rather than negligeance and poorly engineered parts/systems/teams. But among the larger number that did finish you'd have a wider, perhaps more even, spread of results. Ranging from teams that realize that they maybe don't yet have the means, the experience or the development to build a "top ten" car so settle for reliability first and foremost, and the teams that usually do have "top ten" cars, and generally only fail if they push things a little too far (like not enough cooling, or undersized flextures on wishbones).

Again this is just a hypothetical suggestion, and not something I think would actually work at competition.

EDIT-Grammar

[acedeuce802]

I actually like that idea. A design can't be 100% proven by analysis, graphs, numbers, and explanations.

I can't remember the team, but they had carbon half-shafts in the design event (and did well in design, if not won), and then replaced them with steel half shafts after the carbon failed in practice. That would've impacted the design score, had it happened after the dynamic events.

[Charles Kaneb]

I'm pretty pleased with our showing in design. If you can show where the time went during design, build, and test you'll do well. We had what appeared on the surface to be a very simple vehicle - but a tremendous amount of design work and machine time went into it and it showed on the track.

At FSAE West, at least, they have judges who like to see a lot of test results.

[exFSAE]

Wouldn't make a damn bit of difference IMO.

Every team is already designing their cars to comfortably survive endurance. Not like they are intentionally "tuning" the thing through some magic to fail 5 feet past the finish line. Most component or system failures (and resultant DNF's) in my experience are a surprise. You can design a part to a FOS of 1.5-2.0 (immense!!) and have the thing fail the first time you tap the brakes because your FEM was complete crap due to lack of training and experience. I've seen that first hand. That isn't going to change based on whether or not you get judged pre- or post-dynamic events. You're still TRYING to do the same thing just as hard.. whether or not you know what you're doing is a separate issue.

Besides, what from the dynamic events could you use to objectively grade a design? If the car DNF's endurance that's already reflected in the final dynamic score, which then goes into your aggregate. Lap times? Skidpad and accel speeds? That's SO hugely driver dependent at this level that there's no way you could separate the two out.

Besides, I'd say after a few years in industry - particularly motorsport - the judges know when you're hand waving a weak design decision, or when some numbers are made up (like teams claiming to be making more HP than is theoretically possible through a choked flow, 20mm restriction). They may just keep that to themselves or make note of it rather than explicitly calling someone out in the design tent with, "You're full of crap, bro."

Not to mention that after running the cars are going to be extra grimey and dirty which makes it that much more difficult to see things like fabrication quality, fit and finish, etc.

I'll admit when I was in school I thought post-dynamic judging would be a cool thing.. but in retrospect.. makes no difference.

[ZAMR]

I agree. I think the design competition is meant to test how much you know about engineering and physics. Its not about who has designed the fastest and most durable car. Although with knowledge usually comes fastness.

[Paul Achard]

I agree with exFSAE on his points. I might add that design finals happen after the first day of dynamic events (or at least they did at Formula North). This is useful for the judges since at that stage they examine your on-track tuning and diagnostics skills, which they do not focus on in first-round design. Some of the judges were indeed commenting on our car's attitude on track, throttle response, etc, and asking questions to see if we had noticed problems and how we would address them.

[Dunk Mckay]

It certainly shouldn't make a difference, and wouldn't for most teams I hope. I'm not suggesting that the design score be affected by dynamic performance (or lack thereof) but subjectively it probably would, which is why this is just a thought exercise.

One of the reasons (either recently discussed, or I recently read on here) that there's such a high failure rate in FSAE is because there is no threat of heads rolling. People take risks with designs, they try to shave of that much more weight than previous years. Competent or not, if you keep removing material from a part that works eventually it wont, you might go from a safety factor of 2 one year, to 1.75 the next, and 1.5 the next, confident because last year's design didn't show any signs of breaking after all the running and testing it did. And than BANG, left left rear of your car is suddenly dragging along the ground 10 laps into endurance.

Even if marking at design judging is completely objective, you still have to look these professionals, maybe even potential future employers in the eye and go over your design process and why your calcs and tests all said it would work.

Another example would be if you failed because you hadn't done a bolt check and a crucial bolt (are there any that aren't?) came undone. In this case you might not have any trouble justifying your designs, but again you'd have to face these pros/employers knowing that negligence, incompetence or just a mere lapse in concentration has cost you an event. You might just try to make sure you are that much less incompetent/negligent/disorganized, as a team and as individual members of the team.

Note: becoming less incompetent/negligent is also known as studying hard, it's what we students are supposed to be doing, but only maybe half of us really do.

EDIT--spelling 'sigh'

[exFSAE]

<BLOCKQUOTE class="ip-ubbcode-quote"><div class="ip-ubbcode-quote-title">quote:</div><div class="ip-ubbcode-quote-content">One of the reasons (either recently discussed, or I recently read on here) that there's such a high failure rate in FSAE is because there is no threat of heads rolling. People take risks with designs, they try to shave of that much more weight than previous years. </div></BLOCKQUOTE>

From my experience this was just not the case at all (6 years total in one way or another). We were competitive enough to finish in the 20's. And always.. ALWAYS.. the #1 design goal was to have a car that would finish all events and be robust enough to get a lot of testing in before and after the competition.

No threat of heads rolling? Maybe. But after a whole team slaves away for a year (or multiple years) to get a car done, when I was doing design I did NOT want to be THAT guy whose part broke and ruined the whole end experience for everyone.

In my experience - both on our cars and having seen others - the failures and DNF's occurred because of young engineers just not knowing any better. It was never like we had a team of really experienced design engineers with very good predictive tools or instrumentation, and were debating on whether a part design should be FOS = 1.01 or 1.02.

Personally I was probably the most aggressive on FOS and only came in at about 1.25 as a target (which if you comfortably know your max load condition is a HUGE margin for overload). There were guys designing to 2.00 (!!!) and failing parts. Failing them because they didn't understand how to set up and run FEA properly, not knowing the materials all that well, or any number of dumb things. They believed sincerely that they would have a very robust part - like failure wasn't even a thought - and then something comes up and bites them.

And as for critical bolts coming undone.. for one, I've seen enough of tech to know they usually give a pretty good scrutineering for fasteners. And if you fail to properly nut and bolt the thing afterward, that's not a design failure that's a prep failure. That's some inexperienced guy making an honest mistake. Doesn't make the design any more or less legitimate.

Like I said, I understand the viewpoint. I used to share it myself. But.. for the above reasons, I just don't buy it anymore.

[Dunk Mckay]

You're probably more or less right. And from a logical standpoint any team serious about doing well at comp won't do any differently either way. I guess I'm just trying to find a simle way of getting junior inexperienced fsae-ers to think more about overall goals rather than just make the car go fast at any cost, e.g. longevity. There is of course Geoff's "Reasoning your way through.." thread, but telling a self righteous overconfident young engineering student to go and read a thread that works out as about 25 pages of text, and he'll think that you're and idiot and are wasting his time instead of letting him get on and build a racecar the way he wants.

[exFSAE]

That's absolutely a good end goal (getting people to think smarter), but I think in large part it just comes with experience. Get more people involved as freshman and sophomores and by the time they're seniors they'll have some practicality under their belt.

[Claude Rouelle]

If you do not know why you win, you won't know why you lose. If you don't know your strengths, you won't know your weaknesses.

You can have a very good car (and also very good drivers) and even win the competition but if you do not know why your car is good and/or you cannot demonstrate it to the design judges you won't deserve a lot of the 150 design points.

Similarly, you can have an excellent concept, a very good design, an excellent execution and a slight problem can happen in the dynamic events. A little 2 dollars part can stop the whole car but that shouldn't take too much away from the evaluation by the design judge of your car concept, simulation and manufacturing.

The design is the design and the dynamics events are the dynamic events. I feel there is a reason why they are separate. It could be somewhat unfair to the teams to judge the car after the dynamic event; the dynamic events results could influence the design judges. What for example if you have an A+ driver and a B- car?

At the end the design, performance and reliability are invariably connected. Look at last FSAE Lincoln, NE results: most of the design finalists did finish endurance and they finish it very, very well. I don't think it is a coincidence.

[Dunk Mckay]

The idea is that under purely hypothetical conditions the design score was purely objectively given, and not directly influenced by the dynamics. Score would change only due to a change in the students' presentation, so depending on their confidence in the car somewhat. some people still believe they can blag it, and pull the veil over the judges eyes (fools) thinking this way would stop that.

If failure is down to driver error a good engineer will know this and be able to state categorically that the car was not at fault (maybe show how he/she knows this) and move on, other good engineers (judges) shoud recognize this as true. If a £2 part fails it was either under specc'd or the manufacturer didn't make to specification, the latter is extremely unlikely. But again if you have the calculations and data (whcih you should) to back it up and show that it should not have failed then you shouldn't lose anything. If anything it could give you a good opportunity to demonstrate good engineering ability by diagnosing the cause of the failure and where the manufacturer went wrong relative to spec. But like I said the sort of failure down to manufacturer error rather than student error is about as likely as me attending a dinner party with the Queen.

[Zac]

Design (and the other static events) are supposed to test the team not the car.

What would be an interesting addition though would be to have a trained driver take each car out on track for a subjective evaluation.

[Thrainer]

Probably my favourite schedule is to have the first round of Design before the dynamic events and the Design Finals in the evening after the first day of dynamics.

What impresses me is if the team can back up their decisions and designs with testing data and simulation. Stating "0-100 km/h in 3.2 s" is probably good for the Business Presentation. In the Design Event, you better show data and possibly a simulation that predicted this performance already before you started with the manufacturing. You don't necessarily need to have the Acceleration Event before Design, because the teams are expected to bring their own data.

Having problems is part of building a prototype and how the team deals with their problems sets it apart from the others. If e.g. a part failure is analysed, understood and appropriate measures taken, the team should score well.

Regards
Thomas

[Warpspeed]

Seems to me, that design judging is more about how much you know, or have learned about theoretical and applied engineering.
R&D is all about learning from failures, and a truly sound and innovative idea that almost worked is still worthy of a good score.

Experience is a wonderful thing, which is always going to be in fairly short supply among a student team.

Then there is luck and driver skill on the day.

Many different aspects to all of this, which is what makes it all so very interesting.
It's a great way to turn students into engineers, which is the whole idea.
Even if you don't score near the top, what you will have gained will be well worth the long hours, the sweat and tears.

[Z]

<BLOCKQUOTE class="ip-ubbcode-quote"><div class="ip-ubbcode-quote-title">quote:</div><div class="ip-ubbcode-quote-content">Originally posted by exFSAE:
Personally I was probably the most aggressive on FOS and only came in at about 1.25 as a target (which if you comfortably know your max load condition is a HUGE margin for overload). </div></BLOCKQUOTE>
exFSAE,

But, say in the case of suspension design (a common source of DNFs), how do you know how big the bumps are going to be that the car will hit? Not just the bumps at the comp, but also the ones at that old parking lot you might use for testing? Or just how fast will the driver rev the engine before dumping the clutch? And what will be the tyre Cf when that happens?

I reckon FoSs are mainly there to cover for the above sorts of uncertainties. Although also for uncertainties of material qualities, welder's skills, driver's skills, etc., etc...

Or to put it another way, give me any car ever made, and I bet I can break it! http://fsae.com/groupee_common/emoticons/icon_smile.gif

Z

[exFSAE]

<BLOCKQUOTE class="ip-ubbcode-quote"><div class="ip-ubbcode-quote-title">quote:</div><div class="ip-ubbcode-quote-content">Originally posted by Z:
<BLOCKQUOTE class="ip-ubbcode-quote"><div class="ip-ubbcode-quote-title">quote:</div><div class="ip-ubbcode-quote-content">Originally posted by exFSAE:
Personally I was probably the most aggressive on FOS and only came in at about 1.25 as a target (which if you comfortably know your max load condition is a HUGE margin for overload). </div></BLOCKQUOTE>
exFSAE,

But, say in the case of suspension design (a common source of DNFs), how do you know how big the bumps are going to be that the car will hit? Not just the bumps at the comp, but also the ones at that old parking lot you might use for testing? Or just how fast will the driver rev the engine before dumping the clutch? And what will be the tyre Cf when that happens?

I reckon FoSs are mainly there to cover for the above sorts of uncertainties. Although also for uncertainties of material qualities, welder's skills, driver's skills, etc., etc...

Or to put it another way, give me any car ever made, and I bet I can break it! http://fsae.com/groupee_common/emoticons/icon_smile.gif

Z </div></BLOCKQUOTE>

Oh certainly, FOS is there to cover unknowns in materials, manufacturing, and use.

In this case, doing billet aluminum uprights and billet steel hubs... there are minimum mechanical properties an alloy or heat treat spec has to hit, and you can get materials with certs. That took that out of the picture for me. I knew the manufacturing process well and could hit the geometric targets with ease on a CNC.

Then comes load case. To your point, the car was not designed to survive driving over speed bumps and pot holes. We were going to compete at Ford MPG which is dead smooth, and would test at similar venues not cratered like the surface of the moon. So there's a judgement call in there of what you call max load condition, but I thought that was a reasonable assumption to make.

Then even the max condition cases... I think in this case I wheel lift in pure cornering was the highest stress case. We had never lifted wheels in testing or at competition so that was a reasonable target - and certainly 25% over that would have been wild. In any event running a sensitivity study around that point to see which factor the design is most sensitive to and which unknown you really have to design around - I believe the Az was not nearly as sensitive as Ax or certainly Ay with the large bending moments imposed... and if the Ay went much over the design case we'd roll over!

That was all for the upright. I'll be fair and say when I did the hub I think I checked off on the design after the FEM predicted a minimum FOS of 0.98. But that was at a small or single element point at the bottom of a drilled hole in what should have been - by basic principles - a low stress region of the part. The remainder of the thing was 1.20+ or higher. Never wound up having any issues with it and I think that basic design went on for several more cars after I left.

So yes, sometimes you do push the limit http://fsae.com/groupee_common/emoticons/icon_smile.gif Though my earlier statement was that the DNF's come from overlooked things even with a high design FOS. Inexperience or too much blind trust in predictive tools will get you. If you have a good idea of what you're doing and the strengths and limitations of designer-level analysis tools (CosmosWorks) - I think you're just fine.

But we digress...