User:Fabartus/DLS Fix

Fixing the DLS, soonest!

This is a proposal to fix the issues with bad content on the DLS, which is not the bad data itself, but the bad data when downloaded. The solution is like exterminating other vermin, the quickest way is to remove their food supply, so to here I propose we remove the errors with automatic computer data shuffling.

Here's the base N3V survival problem. The new user knows squat about fixing errors, or the perils of downloading a route. The secondary issue, is that those of us who do know that minefield still get stuck with the time-sink of fixing errors when assaying a new route. Since Many routes have dependencies with issues OR, those which have missing dependencies (as bad to my thinking), the goal is to deal with such ASAP as a community, and a corporate partner...

So the strategy is to isolate those with issues, isolate the supporting cast of assets with issues, and isolate those which may have issues... leaving a set of clean problem free assets and routes on the new skinnier DLS that can be safely downloaded by even a third grader without much ado.

THAT resultant will become the new default DLS on a diet. The other stages are user accessible web pages where one can shop and download knowing you may face a risk. In general, new users won't find out about those until they've been around long enough to not get discouraged.

Lastly, I've a number of proposed methods for curing the new backlog, the assets with problems, the missing assets, and so forth. Again, I invoke computer automation with a little software assist from N3V. It's their mess, they benefit most from the attractiveness of the DLS. It's time they supported a system of expedited fixes.

What are those

What are those other websites you ask, how will that work. Some may wonder if it will be expensive. (Compared to alienating customers? Just the opposite.) Well, simple, they're clones of This TRS2004/DLS.php website, just like a page such as this snoop page can be accessed. I presume they'd reside in another harddrive, possibly might need a separate computer, but we aren't talking a large outlay here, a few thousands in equipments should cover it with room to spare. The key difference is they will have a different URL, perhaps TRS2004/DLS-1.php, TRS2004/DLS-2.php, TRS2004/DLS-3.php or some more clever link namings. Whether they are organized as a sub-page, a permanent url, or whatnot I leave to the preferences of N3Vs webmaster and management. The point is there is little hardware and little software needed, there already exists a pipeline into the DLS for vetted and approved assets conformal to V2.9 or above. There already exists an link to the shopping carts in Simulator Central, and capacity to have that shopping cart trigger a download.

Our only requirements in this specification are that

1. they duplicate the search capabilities of the TRS2004 DLS
2. They too can feed the shopping cart for download enqueuing
   1. --preferentially with a different background color to minimize confusion
   2. --they can transfer data to and fro to the other cloned DLS variants at need,
   3. or most importantly, via a channel with filtering supplied by the N3V programmers as discussed below.

Homage

This 'naming' is a homage, a tribute to daytime children's programming when CABLE TV hadn't yet been invented. Don't know how international the Three Stooges reruns were, but the FIVE-to-SEVEN stooges (They had some cast turn over in 45 years) were a vaudeville act gone to Movie Shorts and then Television shorts/programs and ran over four decades as a popular USA entertainment act. 'Larry' was wild haired and 'Curly' was Bald, while 'Moe' had a trademark bowl-snipped hairdo and sometimes sported a Hitleresque mustache but was always the boss except when they debuted in film with Shemp as the principal actor. Their skits relied heavily on abusing the others with eye-pokes and rump kicks, head-knocks and other slapstick comic stuff before society went over soft to the Nanny State motif. Their names are somewhat symbolic of stages in a process aimed to cleaning up the mess on the DLS... quickly

Shemp

Shemp had an successful independent Hollywood and stage career and seen chronologically in their shorts output was a Curly replacement. (Actually Curly replaced Shemp initially—but he returned to the act over a decade later when brother Curly had a series of career and life-ending strokes) so because he's disorderly—with a hairdo from hell—we'll let him represent a clone of the current DLS, warts and all—with one important change:

1. SHEMP will not know what a 'simple KUID' is but input the DLS as it is now—but as all as ALL KUID2 formats^*... with any luck the software gurus will sense the obvious next nicety, a wildcard capability!
2. Shemp normally talks only to Curly, except for two highly constrained specific cases:
   1. unless Larry asks for something,
   2. or Moe tells him what to do.
3. Shemp will not output Class Y category-class assets, Maps, Scenarios, Sessions, except to Larry, but then if and only if Larry asks.
4. When told by Moe, Shemp will delete a map, session, or scenario. Moe does that only because he's received all the assets on the route, session, or scenario from Larry.
    • Signifies a complete ready to run session and route! Yea, team!
    • For those of you following, I sense you sensing that Moe will be holding the new DLS when this all works through.
5. When asked, Shemp will pass other assets to Curly, beginning with simplest most likely to be building block assets, then progressively higher organized kinds of assets.
6. then delete them when Curly says they're received and error checked.
7. Curly will not tell Shemp to delete assets with errors.

Like sand running through an hour=glass, the implication here, good assets, once vetted (and the method assumes this will likely take some time) or repaired leave Curly and Shemp, and in the end, Shemp will end holding only routes with faulty content, Curly only faulty assets.

* Simple kuid's have outlived their usefulness, particularly as they confuse string searching. Since <kuid:WWWXXX:YYYZZZ> can be specified <kuid2:WWWXXX:YYYZZZ:0>, both are redundant. Ironically, once everything is coded for kuid2 formats, I'd advocate (now that the statisticaly population is everything) converting all back to kuids, the 2 then being redundant.

Curly

1. Assume the next Curly is a website you can also stimulate into adding items into a shopping cart, which you can then download. Make the same assumption for each of the others... that downloading of a cart is already in place, n'est pas? Everyone thinking that five pages adding to same cart would overwhelm N3Vs technical capacity, please raise your hand. OK, Shane, you stay after school. <G>
   1. Curly can not get any Sessions, scenarios, or Routes from Shemp, and when Shemp passes the assets on, he must delete them when curly tells him the Kuid2 is OKAY, leaving him holding only Sessions, Routes, and Scenarios--and assets with Errors... (like my CM!)
2. Meanwhile Curly tests the component assets in a prioritized scheme, based on category-class tags and KINDs from simplest to most complicated. More or less, I'd initially think to prioritize searching and input by:
   1. { G= Ground, E= Environment, J= Textures, H= meshes, I= Product, F= Foliage, O= Organisms, V= Vehicles, Z= Train Parts, M=Maintenance Of Way, W= Wayside, T=Track, S= Splines, B= Buildings, A= Motive Power, and rolling stock C,D,L,P,R,X } with a deliberate hold on Y= Maps and scenarios, sessions.
   2. All assets of {the set:
       • Class "A" Motive Power, Class "B" Buildings and Structure, Class "C" Cabees,
       • Class "D" Defence, Class "E" Environment, Class "F" Foliage,
       • Class "G" Ground, Class "H" Mes, Class "I" Product,
       • Class "J" Textures, Class "L" Light Rail & Monorail, Class "M" Maintenance Of Way,
       • Class "O" Organisms, Class "P" Passenger & Mail Car, Class "R" Railcars & Multiple Unit Sets,
       • Class "S" Splines, Class "T" Track, Class "V" Vehicle,
       • Class "W" Wayside, Class "X" Freight Car, Class "Y" Maps and Scenario,
       • Class "Z" Train Parts
      }
3. Those which pass testing for TRS2006-SP1 (upto TC3--higher formats later by Larry) are copied and passed onto LARRY then...
   1. Curly then auto-modifies these substituting an annotation at the end of the description text "This asset auto-updated by DLS automation bot CURLY on 2014-0415, Happy Tax Day! and promoting the KUID2 to the next vacant level. That will usually be '+:1' so <KUID2:WWW:XXX:1> for oldest assets (<KUID2:WWW:XXX:0> is the same as <KUID:WWW:XXX>, lowest numbers most often, trainz-build tag to 2.6. Then passes those also onto Larry.
   2. That leaves Curly Holding a bag full of broken assets
   3. and assets with missing KUIDS (unbroken assets with missing dependencies)... Curly holds onto these until Larry asks for them.

Larry

gets pre-vetted assets needing more TLC

Larry is more clever than Curly as every fan knows. He's going to run the list of KUID2s from Curly and edit the files and folders of v2.6 to v2.8, translating data structures and tags, auto-annotating the description records ... making them TS2009 compliant (v2.9).

Note, that if N4V wants to raise that v2.9 import threshold in the future, this facility will be in place. A little code tweak, and TANE compliant assets could in theory be produced sans our time being spent like our minutes are as cheap as raindrops in the Amazon during a thunderstorm.

1. The software for this is straight forward, for every migration N3V has made in the SIMPLER, OLDER data structures, they've made a more generalized more flexible analog, usually (ALMOST ALWAYS) using containers and/or texture.txt file expansions--EACH OF WHICH extended the flexibility and capabilities of optional asset behaviors.
2. Most of these can be defaulted by Larry as they do and have done for over a decade in their run time (read-in) code now.
   1. Since this is a pipeline translation, one approach would be to write filters, say five or six a day, old format to new (You know and I know they'll steal code from existing source files, so five a day isn't going to take very much of a chunk out of their schedules.), and run those against this DB until you've nothing left to convert.
      1. Filters for those unfamiliar with the concept are short programs using standard I/O facilities which can be 'piped' input to the previous output, and so on in a chain.
      2. Filters have been at the heart of Unix, sysix, linux, C, and even DOS and Window since the IBM PC debut stabilized the home computer market and overnight practically killed off most the operating systems competing with MSDOS (and eventually Windoze).
      3. Since you can chain them, you can also batch them, so bundle a set inside a batch file, selecting apropos sequences of filters to process this KIND of asset or that.
   2. Converted files are promoted with another KUID2 bump up and passed onto Moe, lucky bossy loud, Moe!
      1. One key to this is Moe continues the current vetting of assets to V2.9, whereas in this schema, the current DLS's clone )(SHEMP) would be able to resume accepting upgrades to assets by their own content creators to versions v2.6-v2.8; he passes them onto Curly, and an intermediate KUID (v2.0--V2.8) can resume being entered into the system. Thus partially updated assets can be pulled from our CMs and be fed to the rest of the community. Curly will need to read & run an CMP/TrainzUtil extract to vett for TRS2006-TC3 errors.
      2. Another asset upgrade schema in this thinking is a two way automatic data flow. LARRY requests an asset from his source Curly, but the asset Curly has, has a (potentially insoluable) problem (needing human intervention); meanwhile N3V sets it up so people volunteer and sign up and those accepting get a small add-on utility or patch from N3V which allows N3V to query for a locally modified kuid/asset. Curly initiates this process for <KUID whatever> when our volunteer's CM logs into the site and finds 3-5 users with the asset 'Locally Repaired' (and faultless!).
      3. The system reads three to five such from it's community partners via the tweaked TrainzUtil or whatnot program and compares the 3-5 excluding text babble lines to find three that are matches in the important data definition fields and subfolders. They don't even have to match in username tags--obsolescent tags are trimmed, Photos excluded, as are descriptions, licensing, and the other verbose text tags in the TrainzBaseSpec. A binary comparison finds three matches from community donors, and the asset is replace by the winner under the original name and description, licenses, company, et. al. In the event there is a disagreement (say a texture.txt file on one has an AlphaMask= set to a different state, the program would refer these 'fixed assets' to a human for selection of the best.
3. Folks my brain is fried tonight, so this is the draft promised, albeit unfinished and lacking polish. You should get the gist by now. I'll see if I can finish this by tomorrow afternoon. N3V's in their weekend schedule so a day more won't slow anything down. // FrankB 06:30, 5 April 2014 (UTC)
   1. When Larry's done with basic assets, the only thing left will be things he can't convert, so the humans will go back to write more filters. Very futuristic, eh... Men working for the computer taskmaster! <G>
4. At some point though, the computer will be taken to task by the human slaves and told to start requesting routes from Shemp by starting with scenarios, then sessions, and working on dependent routes.
   1. Assets all accounted for, get passed to Moe as happy faces.
   2. Assets with errors get sent to CURLY! (He was feeling left out of the sessions and routes biz)
   3. Assets with missing dependencies make Larry Sad. He holds onto those, and from time to time checks against the input stream to see if there's news of a missing asset.

MOE

Costs

Now for the price of a few url pages, and a few hard drives, this should be feasible using N3V's servers alone. The general scheme is to divide, conquer, and quarantine. Then update and userfy working content to new standards with apropos kuid advancements.

Some 'asset validation' software will need to be customized by Windwalkr and company, but that should be mostly extracts from prior Trainz released make file resources (source code), and they'll have to work out piping data from server to server (I/O).

Designate Curly to get Content with bad content (errors). Designate MOE to get all vetted and approved ERROR FREE content without missing dependencies

End