> New Upload Servers, Current Stage: Live/Fully Migrated

 
post Sep 26 2021, 10:06
Post #81
Tenboro

Admin




New upload servers have been deployed and can be accessed here: [upld.e-hentai.org] https://upld.e-hentai.org/manage

Note that while this is officially a beta, the servers are live. If something were to break or end up in the wrong state with a gallery during the beta, don't try to "fix" it, leave it and let me know so I can figure out what went wrong.

The new uploaders are now used for new uploads and new versions of existing galleries. Existing galleries will be migrated soon.

The new uploaders are now used for all uploads, and all existing unpublished uploads have been migrated.

--

New uploader servers have now been deployed, which will eventually replace the current uploaders that were deployed six years ago. These new servers also come with a new major release of the uploader interface. While they have been in closed beta testing for the last week or so, I decided to run a public beta as well before switching to the new servers entirely.

The new servers run new major versions of pretty much everything, the internals of the uploader have been significantly reworked, and it also uses the new core library that was first added to the previous version of HV, so please be on the lookout for any issues.

Uploads may temporarily be somewhat slower than before, since files have to also sync over to the old uploaders for this initial release. When the old uploaders have been phased out, this will go back to normal.

Unpublished galleries that were added on the old uploaders will not show up on the new ones until they have been imported, which will happen some time after the new uploaders go fully live. The old uploaders will be available until then.

Similarly, galleries added or versioned (cloned) on the new uploaders will not show up on the old ones, though there was a check added to prevent a gallery from being versioned on both the new and old uploaders.


New Features

- Added time-based auto-sorting that allows you to easily resort a gallery to have newly added files first (or last) without having to manually reorder them, which is a common task for updated image galleries.

This uses the publish time for the gallery in the chain each file first appears in, and is only applicable for versioned (cloned) galleries. Note that the order the files were uploaded in for a given gallery in the chain does not matter for this sort as it goes by which gallery it was first added to, but you can keep the current order within each set of files.

- Added an "auto-sort" method called "Remove Exact Duplicates" which, as the name implies, will remove duplicated files from the gallery. The first file in the current page order will be preserved.

The available methods are now:
* Remove exact duplicate files from the gallery
* Sort based on natural or lexical ordering of the filenames, ignoring when they were added. (These are the old methods.)
* Sort based on the time a file was first added to a chain of galleries, using either a Recent First (descending) or Recent Last (ascending) sort. It can then either keep the current sort order within each set of files, or do an additional natural or lexical sort based on the filenames.


- Added the option to "disown" any published gallery. This will remove the gallery from the uploader interface, remove the uploader link/readout from the gallery, and prevent the gallery from being found with an uploader search.

You will not be able to make (or request) changes to a gallery after disowning it. You will however continue gaining toplist points and GP from disowned galleries.

You cannot disown a gallery before it has been fully published ("Visible" changes from "Soon"). Recently published disowned galleries will still show the uploader for a week after being published.

Note that expunged galleries will now also be "disowned" rather than "deleted". "Deleting" published galleries is never possible anymore. Unpublished galleries will still be erased as before.


- When creating a new gallery, you can now set the initial language for the gallery directly on the uploader screen. This will automatically create Autotagger tags for that language on the gallery when it is published.

For an already published or versioned (cloned) gallery, it will still show the currently active language based on the current tags, but you will not be able to change it from the uploader interface.

Before anyone tries to argue whether a language should be "common" or not, the selection list is dynamically generated and is purely based on the number of galleries using the tag for the language.

To prevent excessive clutter, languages with less than 5 tagged galleries are not included in the list. Any new language tags are automatically added to the list when they reach 5 tagged galleries.

If the category is set to "Image Set", the language is ignored if it is set to "Speechless" or "Text Cleaned". Similarly, if the category is set to "Game CG", the language is ignored if it is set to "Speechless". Note that it allows you to set it, it's just ignored when publishing.


- A new image previewer for unpublished galleries was added, primarily because galleries now only "exist" in the uploader until they are published. This will always use the 1280x resamples if available for the image, originals otherwise.

While previewing an image, you can use right arrow/numpad or D to go to the next page, and left arrow/numpad or A for the previous page. You can also close the image preview with spacebar.

Unlike the preview on the old uploaders, this preview will immediately mirror any file order changes or deletions, so you should be able to quickly check the new page order before saving the changes.


Changes and Fixes

- The way unpublished galleries are stored in the system was fundamentally changed; among other things, this fixes a problem where galleries that were created a long time before they were published would "bypass" the Watched page.

- After publishing, a gallery now have an explicit "processing" state while some final background tasks take place, where they are fully functional except that some thumbnail variants aren't generated yet. They can be accessed (and tagged) if you have the link, but aren't yet publicly listed.

- Thumbnails are now pushed to the standard thumbnail servers immediately rather than as a scheduled background job. The uploader thumbnail servers will be retired together with the old uploaders, and the "Asia delay" will be removed when this is done.

- Reworked some of the mechanisms to prevent excessive expunge posters and new gallery posters from spamming the system. Among other things, this check now happens at publish time rather than gallery creation time.

- You can now create a maximum of 250 upload folders. Note that if you are one of the 18 people who currently have more than this, you might want to consolidate folders manually, as any excess folders may be dropped in a future update, setting those galleries as "Unsorted".

- You can now have a maximum of 1000 unpublished galleries.

- Unpublished galleries will now be deleted 1 year after they were last touched. Older unpublished galleries will have a moratorium of 1 year from the time they are imported to the new system.

- Corrected an issue where restrictive/broken permissions for files in ZIP archives could prevent files from being included. Note that filenames in ZIP files still have to be unique across any subdirectories, otherwise only one of the files with the same filename will be included.

- Support for RAR archive files has been dropped.

- The same exact file can now only appear in a gallery once, mostly to prevent any lingering issues with accidentally starting multiple uploads at the same time. This is not retroactive to old published galleries, but if an old published gallery is cloned and when an old unpublished gallery is imported to the new system, any duplicate files will be removed.

- The same exact file with the same exact filename can now only appear in a gallery once, mostly to prevent any lingering issues with accidentally starting multiple uploads at the same time. This is not retroactive to old published galleries, but if an old published gallery is versioned/cloned, any duplicate files with the same exact filename will be removed; a warning will be printed in this case. Old unpublished galleries will also have any duplicated pages with identical file+name removed when imported to the new system

- When trying to access a replaced gallery in the uploader, it will now forward you to the current (published or unpublished) version of the gallery.

- The readout for "Child Gallery" on the manage screen was never actually used since older versions of a gallery could never be "managed", and was removed.

- "Standard Sort" was renamed "Lexical Sort" to be less ambiguous.

- Galleries that are being updated/replaced are now still shown in the uploader list until the replacement is published, and will have a link to the replacement gallery.

- The category selector was moved, and the "make public" checkbox was removed in favor of a separate option in the selector drop-down.

- Actions that are no longer available on a gallery (such as changing the category or name on non-recently published galleries) are now grayed out instead of just returning an error if you were to try to change them.

- Other minor UI tweaks and fixes.
User is offlineProfile CardPM
Report PostGo to the top of the page
+Quote Post


Posts in this topic
Tenboro   New Upload Servers   Sep 26 2021, 10:06
darkknightx2   I'm so happy to see upgrades :) Thankyou so...   Sep 26 2021, 19:32
notknow   Hope there are some notice in the UI when a unpubl...   Sep 26 2021, 19:42
erana   [quote name='notknow' post='5989008' date='Sep 26 ...   Sep 28 2021, 03:19
Tenboro   I made a small tweak to the gallery locking mechan...   Sep 28 2021, 12:02
dragontamer8740   Cool. Kind of glad my old unpublished tweak of a p...   Sep 27 2021, 16:36
Z.G.   I don't really get the disown (but then again ...   Sep 27 2021, 21:05
Shank   [quote name='Z.G.' post='5989610' date='Sep 27 202...   Sep 27 2021, 21:25
Z.G.   [quote name='Shank' post='5989616' date='Sep 27 20...   Sep 28 2021, 20:54
Deerenbung   Just for my peace of mind, does "the same ex...   Sep 30 2021, 20:14
Tenboro   [quote name='Deerenbung' post='5991050' date='Sep ...   Sep 30 2021, 22:48
Deerenbung   That's concerning to me. (scat warning for th...   Sep 30 2021, 23:29
Tenboro   [quote name='Deerenbung' post='5991139' date='Sep ...   Sep 30 2021, 23:44
Tenboro   Note that I will be taking the beta down in an hou...   Oct 1 2021, 10:42
Shank   Seems to be a little funkiness with zip files I th...   Oct 1 2021, 13:54
Tenboro   [quote name='Shank' post='5991624' date='Oct 1 202...   Oct 1 2021, 14:13
Shank   Ah, I see I'll add some notes about that to th...   Oct 1 2021, 14:45
Tenboro   Another minor change: - When versioning/cloning a...   Oct 2 2021, 10:19
Jo.To   R.I.P RAR it was nice knowing you...we had a good ...   Oct 2 2021, 17:55
Tenboro   Archives in general are kind of obsolete for uploa...   Oct 2 2021, 19:10
nasu   Small visual bug, as a gallery gets larger it seem...   Oct 8 2021, 14:04
Tenboro   [quote name='nasu' post='5995545' date='Oct 8 2021...   Oct 8 2021, 22:51
nasu   [quote name='Tenboro' post='5995768' date='Oct 8 2...   Oct 21 2021, 14:03
Tenboro   [quote name='nasu' post='6002694' date='Oct 21 202...   Oct 21 2021, 22:52
Tenboro   As part of the new uploader rollout, the archivers...   Oct 10 2021, 13:19
Tenboro   I noticed (or rather, the site logged) some potent...   Oct 13 2021, 16:15
JnTo   I hate the fact that you can't go from a page ...   Oct 21 2021, 01:13
Tenboro   [quote name='JnTo' post='6002347' date='Oct 21 202...   Oct 21 2021, 09:48
Tenboro   I added some keyboard navigation to the new upload...   Oct 24 2021, 13:45
Tenboro   The uploaders have now been switched over and will...   Oct 28 2021, 11:20
fabriciodel96   I have a question, it says here that "Unpubli...   Oct 28 2021, 17:39
Tenboro   [quote name='fabriciodel96' post='6006808' date='O...   Oct 28 2021, 18:07
fabriciodel96   [quote name='Tenboro' post='6006814' date='Oct 28 ...   Oct 28 2021, 19:18
okawabi   Is it just me or is there an issue with making fol...   Oct 30 2021, 04:20
Tenboro   [quote name='okawabi' post='6007773' date='Oct 30 ...   Oct 30 2021, 11:09
Hyoros   Few days ago I'm trying to update description ...   Oct 30 2021, 06:48
Tenboro   Older unpublished galleries have now all been migr...   Oct 30 2021, 15:03
fabriciodel96   [quote name='Tenboro' post='6008018' date='Oct 30 ...   Oct 30 2021, 16:49
Tenboro   [quote name='fabriciodel96' post='6008054' date='O...   Oct 30 2021, 19:33
notknow   Phew, good to know about the empty galleries, had ...   Oct 30 2021, 20:43
Yakhont69   got bug where my galleries don't show up in ne...   Oct 30 2021, 20:51
H1dd3nM4st3r   Is there a way to get the /g/ link for my own unpu...   Oct 30 2021, 21:40
Tenboro   [quote name='Yakhont69' post='6008136' date='Oct 3...   Oct 30 2021, 22:18
H1dd3nM4st3r   [quote name='Tenboro' post='6008169' date='Oct 30 ...   Oct 31 2021, 06:49
Noni   [quote name='H1dd3nM4st3r' post='6008363' date='Oc...   Oct 31 2021, 13:09
H1dd3nM4st3r   [quote name='Noni' post='6008473' date='Oct 31 202...   Oct 31 2021, 17:40
nasu   [quote name='H1dd3nM4st3r' post='6008543' date='Oc...   Oct 31 2021, 18:14
H1dd3nM4st3r   [quote name='nasu' post='6008547' date='Oct 31 202...   Oct 31 2021, 18:23
Yakhont69   Thanks, they worked now. Although it took longer t...   Oct 30 2021, 22:56
fabriciodel96   [quote name='Tenboro' post='6008104' date='Oct 30 ...   Oct 30 2021, 23:27
Shank   I don't think its been mentioned yet, and it m...   Oct 31 2021, 18:25
H1dd3nM4st3r   [quote name='Shank' post='6008550' date='Oct 31 20...   Oct 31 2021, 18:29
Shank   As in, if you don't like to use cryptocurrency...   Oct 31 2021, 18:36
H1dd3nM4st3r   [quote name='Shank' post='6008558' date='Oct 31 20...   Oct 31 2021, 18:43
Shank   I'm not familiar with either of those. If it h...   Oct 31 2021, 18:51
H1dd3nM4st3r   [quote name='Shank' post='6008562' date='Oct 31 20...   Oct 31 2021, 19:01
Shank   I can't speak on behalf of tenboro no, and I d...   Oct 31 2021, 19:18
H1dd3nM4st3r   [quote name='Shank' post='6008568' date='Oct 31 20...   Oct 31 2021, 19:22
Jack Woods   [quote name='Shank' post='6008568' date='Oct 31 20...   Nov 1 2021, 06:36
Pangean   I'm really loving these changes but I do with ...   Nov 2 2021, 06:04
Noni   [quote name='Pangean' post='6009333' date='Nov 2 2...   Nov 2 2021, 09:03
Tenboro   The L2 (caching) image servers have now been switc...   Nov 2 2021, 12:54
Tenboro   Minor change: If the category is set to "Imag...   Nov 3 2021, 11:19
Tenboro   The L1 (storage) image servers have now also been ...   Nov 4 2021, 14:28
Tenboro   Image resampling has now been switched from the ol...   Nov 5 2021, 11:06
MSimm1   Bug Report for new Servers The file 0010.png is c...   Nov 6 2021, 21:00
Tenboro   [quote name='MSimm1' post='6011731' date='Nov 6 20...   Nov 6 2021, 23:57
MSimm1   [quote name='Tenboro' post='6011791' date='Nov 6 2...   Nov 8 2021, 19:59
Tenboro   [quote name='MSimm1' post='6012674' date='Nov 8 20...   Nov 8 2021, 20:02
MSimm1   [quote name='Tenboro' post='6012677' date='Nov 8 2...   Nov 10 2021, 20:31
Scumbini   Was the change of default category in the new uplo...   Nov 6 2021, 23:58
Tenboro   [quote name='Scumbini' post='6011792' date='Nov 6 ...   Nov 7 2021, 00:40
Hyoros   " primarily because galleries now only ...   Nov 7 2021, 10:56
Tenboro   The uploader gallery language setting is now ignor...   Nov 7 2021, 11:22
Tenboro   The gallery post-publishing processing, which prim...   Nov 7 2021, 14:03
pork:zero   Is there no quick way to use the "Show all ...   Nov 9 2021, 04:43
Tenboro   [quote name='pork:zero' post='6012869' date='N...   Nov 9 2021, 10:57
pork:zero   [quote name='Tenboro' post='6012965' date='Nov 9 2...   Nov 9 2021, 13:38
Tenboro   [quote name='pork:zero' post='6013012' date='N...   Nov 9 2021, 14:47
Necromusume   http://ehentaihip.com/?f_search=uploader%3ABloobz...   Nov 16 2021, 21:07
Tenboro   [quote name='Necromusume' post='6016657' date='Nov...   Nov 16 2021, 23:21
manongmorcon   Will the preview images in unpublished works not g...   Nov 19 2021, 10:29
Tenboro   [quote name='manongmorcon' post='6018068' date='No...   Nov 19 2021, 12:58
Gochugaru   So I was uploading a gallery today and for some re...   Nov 20 2021, 02:30
Tenboro   [quote name='Gochugaru' post='6018470' date='Nov 2...   Nov 20 2021, 12:10
Soyungundam   I was trying to upload some images and I got the e...   Nov 22 2021, 20:00
Necromusume   $ identify h06vlq.jpg h06vlq.jpg JPEG 1000x14...   Nov 22 2021, 20:53
derpderp2   Thanks for the updates Tenboro :)   Nov 23 2021, 04:18


Closed TopicStart new topic
1 User(s) are reading this topic (0 Guests and 0 Anonymous Users)

 


Lo-Fi Version Time is now: 27th September 2022 - 11:29