| Pages in topic: < [1 2] | Make Directory ranking follow a set of values / weighted average of features Thread poster: traductorchile
|
|---|
Kay Barbara
United Kingdom
Local time: 20:07
Member (2008)
English to German
+ ...
| That's more like it ... but! | Sep 25, 2012 |
Samuel Murray wrote: Sheila Wilson wrote:
I don't know if the OP was thinking of giving more weighting to higher examinations (MA over BA, for example), but even the collection of that information is highly suspect. ... [Can] we expect ProZ.com to decide on the relevancy of a particular qualification from a particular part of the world? And collect all that other data from us, checking on the viability? Well, my support for the original idea was based on the assumption that the aggregate score would be based on things that are already known to ProZ.com. In other words, not anything that requires additional verification. Here is a simple example of what such a scoring system might look like: * WWA entries = 3 points each (maximum 15 points) * Referrals = 2 points each (maximum 10 points) * Number of BB posts = 1 point each (maximum 5 points) * Profile completeness = between 0 and 5 points * Years of experience = 1 point for every 5 years (minimum 0, maximum 5 points) * Number of portfolio translations = 1 point each (maximum 5 points) * And one point each for: - having a downloadable CV/résumé (2 points if more than one language version) - having at least 1 association membership - having at least 100 forum posts from more than 1 year ago - having at least 100 entries in a personal glossary In the above example, the maximum score is 50 and the minimum score is 0. Users can still do things to raise their aggregate scores, but some aspects of the score can't be fixed by the user unless he resorts to deception. Such a score would also help encourage users to participate in the site sections that affect their score. What do you think? I understand the reasoning that someone who has 100 KudoZ points in cosmetics will likely be an experienced cosmetics translator, but not all translators participate in KudoZ, and besides, the KudoZ score is based not on KudoZ participation but on KudoZ luck and KudoZ success. Samuel
When I first read your list, Samuel, I really liked it - it sounded like a neat idea much more so than the OP's suggestions. I found the concept of rewarding people who participate in the community interesting. However, starting with "years of experience" my interest gradually subsided and when I thought about the proposed criteria a bit more, my main concern was that they could pose a barrier for young translators. As I am one from the younger camp, I thought I'd point this out.
Also, Kudoz points are linked to the speciality fields, so how would your point-scheme work depending on which speciality I look for in the directory? And the "years of experience" where, again, anyone is free to claim what they like, just favours older translators regardless of specialisation? Maybe I did get this completely wrong, but now it seems that kudoz points in a speciality field would not matter at all in the directory ranking?
Personally, I have to say that earning kudoz points in order to climb the directory proved to be invaluable. I don't quite know what the "kudoz luck", you referred to, is exactly - but I guess it would be counter-productive to reward kudoz participation alone (the quality of contributions would certainly dip (even more?)). As I said in my earlier post, I would welcome a kudoz ratio (there are a few threads about this matter out there), because I believe this would definitely improve the directory rankings and make them fairer.
[Edited at 2012-09-25 18:57 GMT]
| | | | Samuel Murray 
Netherlands
Local time: 20:07
Member (2006)
English to Afrikaans
+ ...
| The OP's suggestion | Sep 25, 2012 |
Sheila Wilson wrote:
It might be interesting to discuss that, Samuel. But that's a long, long way from the OP's suggestion.
The way I read it, the OP's main point was that KudoZ score should not be the default sorting method, and that the sorting method should be based on some kind of aggregate score. The OP then expanded on his point by suggesting how such a system might work. I agree with his main point, and that is what I've been discussing, but I don't think the system he proposes for it will work (and neither do you), and therefore you'll notice that I didn't respond to any of it.
[Edited at 2012-09-25 17:45 GMT]
| | | | Samuel Murray
Netherlands
Local time: 20:07
Member (2006)
English to Afrikaans
+ ...
| @Kay, some answers | Sep 25, 2012 |
Kay Barbara wrote:
I found the concept of rewarding people who participate in the community interesting. However, starting with "years of experience" my interest gradually subsided and when I thought about the proposed criteria a bit more, my main concern was that they could pose a barrier for young translators.
I don't think such a system would yield valuable results to clients if complete newbie translators can make it to the top of the list by simply tweaking their profile pages. The idea is to put more suitable translators near the top.
Keep in mind that the fraction for experience is just one small part of the whole. The idea is to make a system that is fair to translators without high KudoZ scores and without high experience scores, but without making it useless for clients. If you include only items that would favour the majority of translators, the scores would all become the same.
Maybe I did get this completely wrong, but now it seems that kudoz points in a speciality field would not matter at all in the directory ranking?
Well, my idea is not to replace KudoZ score ranking, but to add another ranking method. If a client believes that KudoZ scores are important indicators of suitable translators, he should select the "Sort by KudoZ scores" option. If he doesn't believe that, and wants to view translators sorted by a more general type of score, he should select the "Sort by aggregate score" option.
I don't quite know what the "KudoZ luck", you referred to, is exactly...
It refers to the combination of pure luck things that result in KudoZ points being awarded to you. Firstly, if 20 translators answer a question, the odds of getting those 4 or 5 points are very small, even if your answer is an excellent answer. By that time it becomes a matter of luck whether you get the points or not. Also, since KudoZ scores are not reset (ever), newer ProZ.com members can never get to the top of the list unless the older members stop answering, leave ProZ.com, or die.
| | | | Samuel Murray
Netherlands
Local time: 20:07
Member (2006)
English to Afrikaans
+ ...
traductorchile wrote:
I think that any option is better than what we have now: anyone can declare a specialty with no supporting evidence at all, as is the same case with "being native" (although this is not part of this thread).
Is this the main point of your suggestion, namely to find a way around the fact that anyone can claim a specialism?
WWA entries= some people don't seek WWA from all clients...
Downloadable CV/résumé = ... many members prefer not to publish a CV...
Association membership = ... I’m sure many top pros don’t belong to any association...
I'm sure there are valid reasons for not filling in any number of sections on the profile page, but ultimately the search results should be useful to a client. And a translator who has a CV and who is a member of an association and who has WWA entries is typically a more suitable translator (from a client's point of view) than one who may have very valid reasons for not offering any of these things. To get a good aggregate score, you have to base it on a variety of things, but the things that tell a client more about general suitability should count more points (hence the higher points for WWA and references, and lower points for things like forum posts or glossaries).
| | |
|
|
|
traductorchile
Chile
Local time: 14:07
English to Spanish
+ ...
TOPIC STARTER
Samuel Murray wrote:
Is this the main point of your suggestion, namely to find a way around the fact that anyone can claim a specialism?
You have it upside down, Samuel. Those who have a way around, are those that claim a specialty, but if challenged cannot show some sort of evidence that would allow them to declare such specialty. I suggest that customers have the right to get a specialist that really is a specialist, or better said, someone that will understand the technical theory and terminology in source and deliver correct messages with the right terminology in the target. I don't know if you've ever had to review/proofread texts in your specialties, but I have seen things that seem written by a child.
And a translator who has a CV and who is a member of an association and who has WWA entries is typically a more suitable translator (from a client's point of view)
This is your personal view, unless you have some statistics of clients’ perceptions. Even still it would be a very erroneous perception, because there is no direct relationship between more suitability and belonging to an association (although it might show that some sort of screening has been performed), neither with showing or not a CV if delivering a CV is an option (and of course, the contents of the CV not its mere existence). WWA is the only point I agree with you, as it shows that those clients were happy with your work, but as I said, many translators don't like to show who all their customers are.
[Edited at 2012-09-25 19:12 GMT]
| | | | Kay Barbara
United Kingdom
Local time: 20:07
Member (2008)
English to German
+ ...
| Expertise in some specialties will be difficult to prove | Sep 25, 2012 |
traductorchile wrote: Samuel Murray wrote:
Is this the main point of your suggestion, namely to find a way around the fact that anyone can claim a specialism? You have it upside down, Samuel. Those who have a way around, are those that claim a specialty, but if challenged cannot show some sort of evidence that would allow them to declare such specialty. I suggest that customers have the right to get a specialist that really is a specialist, or better said, someone that will understand the technical theory and terminology in source and deliver correct messages with the right terminology in the target. I don't know if you've ever had to review/proofread texts in your specialties, but I have seen things that seem written by a child.
And a translator who has a CV and who is a member of an association and who has WWA entries is typically a more suitable translator (from a client's point of view)
This is your personal view, unless you have some statistics of clients’ perceptions. Even still it would be a very erroneous perception, because there is no direct relationship between more suitability and belonging to an association (although it might show that some sort of screening has been performed), neither with showing or not a CV if delivering a CV is an option (and of course, the contents of the CV not its mere existence). WWA is the only point I agree with you, as it shows that those clients were happy with your work, but as I said, many translators don't like to show who all their customers are. [Edited at 2012-09-25 19:12 GMT]
I can relate to the "things that seem written by a child" you have seen - I have seen them too. And I am certain that it could have been avoided if a true specialist had been in charge of the translation job. I am confident that I would have been better qualified for that particular translation I once saw but how am I supposed to "prove" my expertise in this field (unfortunately I never received a degree or proficiency certificate for playing games )? Even though I would definitely like to see fewer untrue/exaggerated claims of specialties, proving some of them may prove to be difficult.
Lastly, I also agree that belonging to a membership doesn't necessarily tell you anything interesting about a translator and there are good reasons why I am not a member - so why would I have an disadvantage compared to someone who is a member?
| | | | traductorchile
Chile
Local time: 14:07
English to Spanish
+ ...
TOPIC STARTER
Kay Barbara wrote: (unfortunately I never received a degree or proficiency certificate for playing games )?
Most organizations accept that if you have evidence of a good amount of translation work in that field that would amount to expertise. Ideally, with repeated work from the, or some, same customers.
Evidence IMHO = legal invoices, published work, work contract, etc.
Maybe, you as an expert in gaming can provide an objective means to differentiate an expert from a novice, something that is not beating the other at the game?
| | | | traductorchile
Chile
Local time: 14:07
English to Spanish
+ ...
TOPIC STARTER | Implementation | Oct 3, 2012 |
Here I summarize what I consider are the advantages,
disadvantages and ease of implementation.
Desired outcome; when someone searches in the directory by:
A specialty: Only translators that have verified that specialty appear in the list, and they are ranked by their average points.
Expertise: A client can still chose if he prefers someone non-specialist or self-defined as “working” or “interest”, overriding the default specialty ranking.
Of course, all this will require SQL statements and some coding, obviously quite a bit of work, but change doesn’t come free, and if proz.com wants’ to stand out as the best translators workplace (as some have mentioned) it should clean up some gaps that don’t help customers when they search for the right translator for the job. One of these gaps is that anyone with no background whatsoever can appear as a specialist in any field, and in many fields. There is no minimum requirement to declare a specialty and worse of all, there is an artificial procedure for lifting translators into specialties. Many Kudoz collaborators have witnessed many of the biases that occur: some have to do with mistakes done by askers, others due to those who answer, other due to peer review, and others due to proz systems, and wrong answers are only a small part of the problem. The worst of all is that in the growing perception of customers, Kudoz doesn’t reflect what it shows in the directory, making the directory an increasingly invalid means of searching for competent translators.
In fact many translators get work because they were found through the directory, not by bidding. If the directory doesn’t reflect specialty competence customers will use it with distrust or suspicion, and competent translators will start considering if they should drop their membership.
[Edited at 2012-10-03 05:44 GMT]
| | |
|
|
|
traductorchile
Chile
Local time: 14:07
English to Spanish
+ ...
TOPIC STARTER | Wrong information doesn't help to find the right translators | Nov 15, 2012 |
Proz.com has declined changing the Directory ranking criteria in its response to the above Poll in record time, less than a month, while other polls are still under observation after months or years. In this way Proz.com staff have denied its members the opportunity to evaluate, cogitate and take a stance regarding an issue that is clearly complex and fairly important: Does Proz.com deliver the correct information for outsourcers to take correct decisions?
Why does Proz do this? ... See more Proz.com has declined changing the Directory ranking criteria in its response to the above Poll in record time, less than a month, while other polls are still under observation after months or years. In this way Proz.com staff have denied its members the opportunity to evaluate, cogitate and take a stance regarding an issue that is clearly complex and fairly important: Does Proz.com deliver the correct information for outsourcers to take correct decisions?
Why does Proz do this? I can’t know because I can’t read other peoples’ minds, so I can only suppose that it is convenient for Proz to keep things as they are, and so, dispose of anything inconvenient (for Proz’ interests) as quickly as possible. In its argument Proz states something about search parameters, but anyone knows that the search parameters have nothing to do with ranking, so the argument is null, that is, the root of the problem is not addressed.
The mere word “ranking” means: “Position on a scale in relation to others”, that is, what people expect of a ranking is that it delivers their best option at the top and their least best option at the bottom, unless, of course, they should reverse the ranking.
I am convinced, and I know others have the same view, that Kudoz tends to deliver deceitful information about the real specialty competence of translators in the directory so it can fool many clients that are not aware of the biases involved.
If Proz.com would have rejected the proposal, recognizing that there are biases, but as it is too costly or complex to modify, they prefer to keep a system that is inadequate instead of having no system, I might have considered their answer as understandable. Not sustainable, but understandable. At least outsourcers would have a clear idea of how they should evaluate the ranking results of the Directory. ▲ Collapse
| | | | | Pages in topic: < [1 2] | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Make Directory ranking follow a set of values / weighted average of features | Trados Studio 2022 Freelance | The leading translation software used by over 270,000 translators.
Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop
and cloud solution, empowering you to work in the most efficient and cost-effective way.
More info » |
| | CafeTran Espresso | You've never met a CAT tool this clever!
Translate faster & easier, using a sophisticated CAT tool built by a translator / developer.
Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools.
Download and start using CafeTran Espresso -- for free
Buy now! » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
