Spaces:
Running
on
Zero
Running
on
Zero
amlpai04
commited on
Commit
·
15a60a2
1
Parent(s):
5eb89c4
Norg support?
Browse files- app/content/NOR/htrflow/htrflow_col1.md +18 -0
- app/content/NOR/htrflow/htrflow_col2.md +23 -0
- app/content/NOR/htrflow/htrflow_row1.md +3 -0
- app/content/NOR/htrflow/htrflow_tab1.md +7 -0
- app/content/NOR/htrflow/htrflow_tab2.md +7 -0
- app/content/NOR/htrflow/htrflow_tab3.md +7 -0
- app/content/NOR/htrflow/htrflow_tab4.md +7 -0
- app/main.py +14 -4
- app/tabs/adv_htrflow_tab.py +2 -1
- app/tabs/htrflow_tab.py +1 -1
- pyproject.toml +2 -2
- uv.lock +0 -0
app/content/NOR/htrflow/htrflow_col1.md
ADDED
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
### Introduktion
|
2 |
+
|
3 |
+
Riksarkivet presenterar en demonstrationspipeline för HTR (Handwritten Text Recognition). Pipelinen består av två instanssegmenteringsmodeller: en tränad för att segmentera textregioner i bilder av löpande-textdokument och en annan tränad för att segmentera textrader inom dessa regioner. Textraderna transkriberas därefter av en textigenkänningsmodell som är tränad på ett stort dataset med svensk handskrift från 1600- till 1800-talet.
|
4 |
+
|
5 |
+
### Användning
|
6 |
+
|
7 |
+
Det är viktigt att betona att denna applikation främst är avsedd för demonstrationsändamål. Målet är att visa upp vår pipeline för att transkribera historiska dokument med löpande text, inte att använda pipelinen i storskalig produktion.
|
8 |
+
**Obs**: I framtiden kommer vi att optimera koden för att passa ett produktionsscenario med multi-GPU och batch-inferens, men detta arbete pågår fortfarande. <br>
|
9 |
+
|
10 |
+
För en inblick i de kommande funktionerna vi arbetar med:
|
11 |
+
|
12 |
+
- Navigera till > **Översikt** > **Ändringslogg och roadmap**.
|
13 |
+
|
14 |
+
### Begränsningar
|
15 |
+
|
16 |
+
Demon, som är värd på Huggingface och tilldelad en T4 GPU, kan bara hantera två användarinlämningar åt gången. Om du upplever långa väntetider eller att applikationen inte svarar, är detta anledningen. I framtiden planerar vi att själva vara värdar för denna lösning, med en bättre server för en förbättrad användarupplevelse, optimerad kod och flera modellalternativ. Spännande utveckling är på gång!
|
17 |
+
|
18 |
+
Det är också viktigt att notera att modellerna fungerar på löpande text och inte text i tabellformat.
|
app/content/NOR/htrflow/htrflow_col2.md
ADDED
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
## Source Code
|
2 |
+
|
3 |
+
Please fork and leave a star on Github if you like it! The code for this project can be found here:
|
4 |
+
|
5 |
+
- [Github](https://github.com/Riksarkivet/HTRFLOW)
|
6 |
+
|
7 |
+
**Note**: We will in the future package all of the code for mass HTR (batch inference on multi-GPU setup), but the code is still work in progress.
|
8 |
+
|
9 |
+
## Models
|
10 |
+
|
11 |
+
The models used in this demo are very much a work in progress, and as more data, and new architectures, becomes available, they will be retrained and reevaluated. For more information about the models, please refer to their model-cards on Huggingface.
|
12 |
+
|
13 |
+
- [Riksarkivet/rtmdet_regions](https://huggingface.co/Riksarkivet/rtmdet_regions)
|
14 |
+
- [Riksarkivet/rtmdet_lines](https://huggingface.co/Riksarkivet/rtmdet_lines)
|
15 |
+
- [Riksarkivet/satrn_htr](https://huggingface.co/https://huggingface.co/Riksarkivet/satrn_htr)
|
16 |
+
|
17 |
+
## Datasets
|
18 |
+
|
19 |
+
Train and testsets created by the Swedish National Archives will be released here:
|
20 |
+
|
21 |
+
- [Riksarkivet/placeholder_region_segmentation](https://huggingface.co/datasets/Riksarkivet/placeholder_region_segmentation)
|
22 |
+
- [Riksarkivet/placeholder_line_segmentation](https://huggingface.co/datasets/Riksarkivet/placeholder_line_segmentation)
|
23 |
+
- [Riksarkivet/placeholder_htr](https://huggingface.co/datasets/Riksarkivet/placeholder_htr)
|
app/content/NOR/htrflow/htrflow_row1.md
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
## The Pipeline in Overview
|
2 |
+
|
3 |
+
The steps in the pipeline can be seen below as follows:
|
app/content/NOR/htrflow/htrflow_tab1.md
ADDED
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
### Binarization
|
2 |
+
|
3 |
+
The reason for binarizing the images before processing them is that we want the models to generalize as well as possible. By training on only binarized images and by binarizing images before running them through the pipeline, we take the target domain closer to the training domain, and reduce negative effects of background variation, background noise etc., on the final results. The pipeline implements a simple adaptive thresholding algorithm for binarization.
|
4 |
+
|
5 |
+
<figure>
|
6 |
+
<img src="https://github.com/Borg93/htr_gradio_file_placeholder/blob/main/app_project_bin.png?raw=true" alt="HTR_tool" style="width:70%; display: block; margin-left: auto; margin-right:auto;" >
|
7 |
+
</figure>
|
app/content/NOR/htrflow/htrflow_tab2.md
ADDED
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
### Text-region segmentation
|
2 |
+
|
3 |
+
To facilitate the text-line segmentation process, it is advantageous to segment the image into text-regions beforehand. This initial step offers several benefits, including reducing variations in line spacing, eliminating blank areas on the page, establishing a clear reading order, and distinguishing marginalia from the main text. The segmentation model utilized in this process predicts both bounding boxes and masks. Although the model has the capability to predict both, only the masks are utilized for the segmentation tasks of lines and regions. An essential post-processing step involves checking for regions that are contained within other regions. During this step, only the containing region is retained, while the contained region is discarded. This ensures that the final segmented text-regions are accurate and devoid of overlapping or redundant areas. This ensures that there’s no duplicate text-regions sent to the text-recognition model.
|
4 |
+
|
5 |
+
<figure>
|
6 |
+
<img src="https://github.com/Borg93/htr_gradio_file_placeholder/blob/main/app_project_region.png?raw=true" alt="HTR_tool" style="width:70%; display: block; margin-left: auto; margin-right:auto;" >
|
7 |
+
</figure>
|
app/content/NOR/htrflow/htrflow_tab3.md
ADDED
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
### Text-line segmentation
|
2 |
+
|
3 |
+
This is also an instance segmentation model, trained on extracting text-lines from the cropped text-regions. The same post-processing as in the text-region segmentation step, is done in the text-line segmentation step.
|
4 |
+
|
5 |
+
<figure>
|
6 |
+
<img src="https://github.com/Borg93/htr_gradio_file_placeholder/blob/main/app_project_line.png?raw=true" alt="HTR_tool" style="width:70%; display: block; margin-left: auto; margin-right:auto;" >
|
7 |
+
</figure>
|
app/content/NOR/htrflow/htrflow_tab4.md
ADDED
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
### Text Recognition
|
2 |
+
|
3 |
+
The text-recognition model was trained on approximately one million handwritten text-line images ranging from the 17th to the 19th century. See the model card for detailed evaluation results, and results from some fine-tuning experiments.
|
4 |
+
|
5 |
+
<figure>
|
6 |
+
<img src="https://github.com/Borg93/htr_gradio_file_placeholder/blob/main/app_project_htr.png?raw=true" alt="HTR_tool" style="width:70%; display: block; margin-left: auto; margin-right:auto;" >
|
7 |
+
</figure>
|
app/main.py
CHANGED
@@ -23,12 +23,19 @@ LANG_CHOICES = ["ENG", "SWE"]
|
|
23 |
|
24 |
with gr.Blocks(title="HTRflow", theme=theme, css=css) as demo:
|
25 |
with gr.Row():
|
26 |
-
local_language = gr.BrowserState(
|
|
|
|
|
27 |
main_language = gr.State(value="ENG")
|
28 |
|
29 |
with gr.Column(scale=1):
|
30 |
language_selector = gr.Dropdown(
|
31 |
-
choices=LANG_CHOICES,
|
|
|
|
|
|
|
|
|
|
|
32 |
)
|
33 |
|
34 |
with gr.Column(scale=2):
|
@@ -52,7 +59,10 @@ with gr.Blocks(title="HTRflow", theme=theme, css=css) as demo:
|
|
52 |
with gr.Tab(label="Data Explorer") as tab_data_explorer:
|
53 |
data_explorer.render()
|
54 |
|
55 |
-
@demo.load(
|
|
|
|
|
|
|
56 |
def load_language(saved_values):
|
57 |
return (saved_values,) * 3
|
58 |
|
@@ -86,4 +96,4 @@ with gr.Blocks(title="HTRflow", theme=theme, css=css) as demo:
|
|
86 |
demo.queue()
|
87 |
|
88 |
if __name__ == "__main__":
|
89 |
-
demo.launch(server_name="0.0.0.0", server_port=
|
|
|
23 |
|
24 |
with gr.Blocks(title="HTRflow", theme=theme, css=css) as demo:
|
25 |
with gr.Row():
|
26 |
+
local_language = gr.BrowserState(
|
27 |
+
default_value="ENG", storage_key="selected_language"
|
28 |
+
)
|
29 |
main_language = gr.State(value="ENG")
|
30 |
|
31 |
with gr.Column(scale=1):
|
32 |
language_selector = gr.Dropdown(
|
33 |
+
choices=LANG_CHOICES,
|
34 |
+
value="ENG",
|
35 |
+
container=False,
|
36 |
+
min_width=50,
|
37 |
+
scale=0,
|
38 |
+
elem_id="langdropdown",
|
39 |
)
|
40 |
|
41 |
with gr.Column(scale=2):
|
|
|
59 |
with gr.Tab(label="Data Explorer") as tab_data_explorer:
|
60 |
data_explorer.render()
|
61 |
|
62 |
+
@demo.load(
|
63 |
+
inputs=[local_language],
|
64 |
+
outputs=[language_selector, main_language, overview_language],
|
65 |
+
)
|
66 |
def load_language(saved_values):
|
67 |
return (saved_values,) * 3
|
68 |
|
|
|
96 |
demo.queue()
|
97 |
|
98 |
if __name__ == "__main__":
|
99 |
+
demo.launch(server_name="0.0.0.0", server_port=7861, enable_monitoring=False)
|
app/tabs/adv_htrflow_tab.py
CHANGED
@@ -9,6 +9,7 @@ with gr.Blocks() as adv_htrflow_pipeline:
|
|
9 |
# TODO: For the viewer we should be able to select from the output of the model what for values we want to
|
10 |
# TODO: add batch predictions here..
|
11 |
# TODO: add load from s3, local, hf daasets( however everything could go through hf_datasets).
|
|
|
12 |
|
13 |
image_mask = gr.ImageMask(interactive=True)
|
14 |
|
@@ -22,7 +23,7 @@ with gr.Blocks() as adv_htrflow_pipeline:
|
|
22 |
interactive=True,
|
23 |
)
|
24 |
test = gr.Dropdown( # TODO: This should be a dropdown to decide input image or mask or s3 or local path
|
25 |
-
["Upload", "Draw", "s3", "local"],
|
26 |
value="Upload",
|
27 |
multiselect=False,
|
28 |
label="Upload method",
|
|
|
9 |
# TODO: For the viewer we should be able to select from the output of the model what for values we want to
|
10 |
# TODO: add batch predictions here..
|
11 |
# TODO: add load from s3, local, hf daasets( however everything could go through hf_datasets).
|
12 |
+
# TODO: send a crop from the user
|
13 |
|
14 |
image_mask = gr.ImageMask(interactive=True)
|
15 |
|
|
|
23 |
interactive=True,
|
24 |
)
|
25 |
test = gr.Dropdown( # TODO: This should be a dropdown to decide input image or mask or s3 or local path
|
26 |
+
["Upload", "Draw", "s3", "local", "crop"],
|
27 |
value="Upload",
|
28 |
multiselect=False,
|
29 |
label="Upload method",
|
app/tabs/htrflow_tab.py
CHANGED
@@ -141,7 +141,7 @@ with gr.Blocks() as htrflow_pipeline:
|
|
141 |
with gr.Accordion(label="Pipeline", open=False):
|
142 |
with gr.Row() as simple_pipeline:
|
143 |
with gr.Column():
|
144 |
-
with gr.Row():
|
145 |
simple_segment_model = gr.Textbox(
|
146 |
"model1", label="Segmentation", info="Info about the Segmentation model"
|
147 |
)
|
|
|
141 |
with gr.Accordion(label="Pipeline", open=False):
|
142 |
with gr.Row() as simple_pipeline:
|
143 |
with gr.Column():
|
144 |
+
with gr.Row(): # TODO: use dynamic rendering instead to make it more clean :https://www.youtube.com/watch?v=WhAMvOEOWJw&ab_channel=HuggingFace
|
145 |
simple_segment_model = gr.Textbox(
|
146 |
"model1", label="Segmentation", info="Info about the Segmentation model"
|
147 |
)
|
pyproject.toml
CHANGED
@@ -17,9 +17,9 @@ classifiers = [
|
|
17 |
requires-python = ">=3.10,<3.13"
|
18 |
|
19 |
dependencies = [
|
20 |
-
"torch==2.0.1",
|
21 |
"htrflow==0.1.3",
|
22 |
-
"gradio>=5.
|
23 |
"datasets>=3.2.0",
|
24 |
"pandas>=2.2.3",
|
25 |
"jinja2>=3.1.4",
|
|
|
17 |
requires-python = ">=3.10,<3.13"
|
18 |
|
19 |
dependencies = [
|
20 |
+
# "torch==2.0.1",
|
21 |
"htrflow==0.1.3",
|
22 |
+
"gradio>=5.11.0",
|
23 |
"datasets>=3.2.0",
|
24 |
"pandas>=2.2.3",
|
25 |
"jinja2>=3.1.4",
|
uv.lock
CHANGED
The diff for this file is too large to render.
See raw diff
|
|