{"id":1236,"date":"2025-02-21T17:20:23","date_gmt":"2025-02-21T16:20:23","guid":{"rendered":"https:\/\/it.ldf.mendelu.cz\/?p=1236"},"modified":"2025-02-25T15:37:57","modified_gmt":"2025-02-25T14:37:57","slug":"hromadna-komprese-dat-z-fotopasti","status":"publish","type":"post","link":"https:\/\/it.ldf.mendelu.cz\/?p=1236","title":{"rendered":"Hromadn\u00e1 komprese dat z fotopast\u00ed"},"content":{"rendered":"\n

Je mi velkou ct\u00ed ozn\u00e1mit, \u017ee d\u00edky vy\u0159azen\u00fdm, av\u0161ak st\u00e1le v\u00fdkonn\u00fdm server\u016fm od Seznam.cz m\u016f\u017eeme v\u00fdrazn\u011b zefektivnit pr\u00e1ci na\u0161ich v\u011bdeck\u00fdch pracovn\u00edk\u016f. Tito odborn\u00edci sb\u00edraj\u00ed data z fotopast\u00ed v les\u00edch a n\u00e1sledn\u011b je analyzuj\u00ed pro sv\u00e9 v\u00fdzkumy. Mo\u017en\u00e1 se to zd\u00e1 jako banalita, ale sesb\u00edrat des\u00edtky terabajt\u016f dat ze stovek r\u016fzn\u00fdch fotopast\u00ed a kamer p\u0159ipevn\u011bn\u00fdch na divok\u00e1 prasata je n\u00e1ro\u010dn\u00fd \u00fakol. Tyto obrovsk\u00e9 objemy dat je nutn\u00e9 nejen bezpe\u010dn\u011b skladovat, ale tak\u00e9 efektivn\u011b zpracovat, aby si zachovaly svou informa\u010dn\u00ed hodnotu a z\u00e1rove\u0148 zab\u00edraly m\u00e9n\u011b m\u00edsta na diskov\u00fdch pol\u00edch.<\/p>\n\n\n\n

Proto jsem vyvinul automatiza\u010dn\u00ed n\u00e1stroj pro hromadnou kompresi fotografi\u00ed a dat, kter\u00fd b\u011b\u017e\u00ed na na\u0161ich serverech p\u0159es noc. V\u011bdci tak mohou b\u011bhem dne nahr\u00e1t i n\u011bkolik terabajt\u016f fotografi\u00ed, kter\u00e9 jsou pot\u00e9 automaticky zpracov\u00e1ny p\u0159es noc. R\u00e1no pak maj\u00ed k dispozici optimalizovan\u00e1 data p\u0159ipraven\u00e1 k dal\u0161\u00ed anal\u00fdze.<\/p>\n\n\n\n

D\u00edky pe\u010dliv\u00e9mu v\u00fdb\u011bru kompresn\u00edch algoritm\u016f se mi poda\u0159ilo sn\u00ed\u017eit velikost dat v pr\u016fm\u011bru o 62,9 % – 65,7%, a to bez viditeln\u00e9 ztr\u00e1ty kvality. Syst\u00e9m je nav\u00edc kompletn\u011b automatizovan\u00fd a nevy\u017eaduje \u017e\u00e1dn\u00fd z\u00e1sah ze strany v\u011bdc\u016f. Ti jednodu\u0161e nahraj\u00ed sv\u00e1 data a nejpozd\u011bji druh\u00fd den maj\u00ed k dispozici jejich komprimovanou verzi.<\/p>\n\n\n\n

Kompresn\u00ed proces je vysoce efektivn\u00ed \u2013 data jsou shroma\u017e\u010fov\u00e1na do fronty a zpracov\u00e1v\u00e1na sou\u010dasn\u011b a\u017e 24 procesorov\u00fdmi vl\u00e1kny. Na\u0161e servery, osazen\u00e9 procesory s TDP 60 W, zvl\u00e1daj\u00ed tento \u00fakol s vysokou efektivitou a n\u00edzkou spot\u0159ebou energie.<\/p>\n\n\n\n

Tento projekt je skv\u011bl\u00fdm p\u0159\u00edkladem toho, jak lze efektivn\u011b vyu\u017e\u00edt zd\u00e1nliv\u011b zastaral\u00fd hardware k podpo\u0159e v\u011bdeck\u00e9ho v\u00fdzkumu. Jsem p\u0159esv\u011bd\u010den, \u017ee v budoucnu dok\u00e1\u017ei tento syst\u00e9m je\u0161t\u011b zdokonalit a nab\u00eddnout v\u011bdc\u016fm je\u0161t\u011b lep\u0161\u00ed podm\u00ednky pro jejich pr\u00e1ci.<\/p>\n\n\n\n

Jak na to<\/h2>\n\n\n
\n
\"\"
Pokud jste doma nebo na Wi-Fi eduroam, p\u0159ipojte se na \u0161koln\u00ed VPN. \u0161koln\u00ed VPN funguje i na androidu. Pokud jste p\u0159ipojen\u00ed v dr\u00e1tov\u00e9 s\u00edti univerzity, \u0161koln\u00ed VPN NEPOT\u0158EBUJETE<\/strong>.<\/figcaption><\/figure>\n<\/div>\n\n\n

S programem datoles<\/a> si v univerzitn\u00ed s\u00edti v\u011bdci p\u0159ipoj\u00ed datov\u00e9 \u00falo\u017ei\u0161t\u011b k PC, ze kter\u00e9ho budou nahr\u00e1vat fotografie:<\/p>\n\n\n

\n
\"\"
Zvolte jazyk – choose your language<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"
D\u00e1le zvol\u00ed sv\u016fj univerzitn\u00ed login, vygenerovan\u00e9 heslo, n\u00e1zev serveru a n\u00e1zev s\u00ed\u0165ov\u00e9 slo\u017eky a disk se jim automaticky p\u0159ipoj\u00ed. Nemus\u00ed n\u00e1sledovat \u017e\u00e1dn\u00e9 slo\u017eit\u00e9 n\u00e1vody, sna\u017e\u00edme se v\u011bdc\u016fm v u\u017eivatelsk\u00e9 jednoduchosti j\u00edt co nejv\u00edce naproti.<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"
Uk\u00e1zka, kdy\u017e v\u0161e klapne a program v\u00e1s p\u0159ipoj\u00ed<\/figcaption><\/figure>\n<\/div>\n\n\n

V Tento po\u010d\u00edta\u010d pak u\u017e uvid\u00edte s\u00ed\u0165ov\u00fd disk:<\/p>\n\n\n

\n
\"\"
Uk\u00e1zka s\u00ed\u0165ov\u00e9ho disku, kter\u00fd je pravideln\u011b skenov\u00e1n ClamAV antivirem a ve kter\u00e9m se nach\u00e1z\u00ed slo\u017eky, do kter\u00fdch ka\u017ed\u00fd u\u017eivatel nahraje data z fotopast\u00ed <\/figcaption><\/figure>\n<\/div>\n\n\n
\"\"
Do slo\u017eky sem_nahraj v\u011bdec nahraje data z fotopast\u00ed a n\u00e1sleduj\u00edc\u00ed den se mu v\u0161echna data objev\u00ed v adres\u00e1\u0159i zkomprimovane. <\/figcaption><\/figure>\n\n\n\n

Otestov\u00e1no<\/h2>\n\n\n\n

ze 1443 MB server dok\u00e1zal hromadn\u011b zkomprimovat data z fotopast\u00ed na 494 MB. <\/p>\n\n\n

\n
\"\"
Uk\u00e1zka uspo\u0159\u00e1d\u00e1n\u00ed slozek pro v\u011bdce<\/figcaption><\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n

\n
\"\"
P\u016fvodn\u00ed velikost testovac\u00ed dat<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"
Zkomprimvoan\u00e1 velikost testovac\u00edch dat<\/figcaption><\/figure>\n<\/div>\n\n\n

Z\u00e1lohovac\u00ed syst\u00e9m<\/h2>\n\n\n\n

Sou\u010d\u00e1st\u00ed \u0159e\u0161en\u00ed budou p\u0159\u00edr\u016fstkov\u00e9 z\u00e1lohy na dal\u0161\u00ed stroje pro dlouhodob\u00e9 skladov\u00e1n\u00ed dat. Zat\u00edm v p\u0159\u00edprav\u011b.
Nyn\u00ed v\u0161ak ji\u017e funguje replikace ve\u0161ker\u00fdch dat na druh\u00e9 diskov\u00e9 pole na jin\u00e9m stroji.<\/p>\n","protected":false},"excerpt":{"rendered":"

Je mi velkou ct\u00ed ozn\u00e1mit, \u017ee d\u00edky vy\u0159azen\u00fdm, av\u0161ak st\u00e1le v\u00fdkonn\u00fdm server\u016fm od Seznam.cz m\u016f\u017eeme v\u00fdrazn\u011b zefektivnit pr\u00e1ci na\u0161ich v\u011bdeck\u00fdch pracovn\u00edk\u016f. Tito odborn\u00edci sb\u00edraj\u00ed data z fotopast\u00ed v les\u00edch a n\u00e1sledn\u011b je analyzuj\u00ed pro sv\u00e9 v\u00fdzkumy. Mo\u017en\u00e1 se to zd\u00e1 jako banalita, ale sesb\u00edrat des\u00edtky terabajt\u016f dat ze stovek r\u016fzn\u00fdch fotopast\u00ed a kamer p\u0159ipevn\u011bn\u00fdch na … <\/p>\n

Pokra\u010dovat ve \u010dten\u00ed „Hromadn\u00e1 komprese dat z fotopast\u00ed“<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":1252,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[58],"tags":[98,103],"class_list":["post-1236","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technovinky","tag-datoles","tag-fotopasti"],"_links":{"self":[{"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/posts\/1236","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1236"}],"version-history":[{"count":11,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/posts\/1236\/revisions"}],"predecessor-version":[{"id":1258,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/posts\/1236\/revisions\/1258"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=\/wp\/v2\/media\/1252"}],"wp:attachment":[{"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1236"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1236"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/it.ldf.mendelu.cz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1236"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}