# DQ for Python API
[](https://pypi.python.org/pypi/dq-batch-client) [](https://github.com/Algolytics/dq_batch_client/actions/workflows/build.yml) [](https://dq-batch-client.readthedocs.io/en/latest/?badge=latest)
## Python client for dataquality.pl
Python library which allows to use [http://dataquality.pl](http://dataquality.pl/) in the easy way.
* Free software: Apache Software License 2.0
* Documentation: [https://dq-batch-client.readthedocs.io](https://dq-batch-client.readthedocs.io/).
### Features
* Full API client
* Automatic encoding file conversion
### Credits
This package was created by [Algolytics](http://algolytics.com/) dev team.
## Installation
### Requirements
For the full functionality Python 3 is required.
### Stable release
To install Python client for dataquality.pl, run this command in your terminal:
```
$ pip install dq-batch-client
```
This is the preferred method to install Python client for dataquality.pl, as it will always install the most recent stable release.
If you don’t have [pip](https://pip.pypa.io/) installed, this [Python installation guide](http://docs.python-guide.org/en/latest/starting/installation/) can guide you through the process.
### From sources
The sources for Python client for dataquality.pl can be downloaded from the [Github repo](https://github.com/Algolytics/dq_batch_client).
You can either clone the public repository:
```
$ git clone git://github.com/Algolytics/dq_batch_client
```
Or download the [tarball](https://github.com/Algolytics/dq_batch_client/tarball/master):
```
$ curl -OL https://github.com/Algolytics/dq_batch_client/tarball/master
```
Once you have a copy of the source, you can install it with:
```
$ python setup.py install
```
## Usage
To use Python client for dataquality.pl in a project:
```
from dq import DQClient, JobConfig
dq = DQClient('https://app.dataquality.pl', user='', token='')
```
API token can be obtain on the page “Moje konto”.
### Account
Check account status:
```
account = dq.account_status()
print(account.email) # user email
print(account.balance) # account balance
print(account.total_records) # processed records
```
### Jobs
#### List jobs
```
jobs = dq.list_jobs()
for job in jobs:
print(job.id) # job id
print(job.name) # human readable job name
print(job.status) # job status
print(job.start_date) # job start date
print(job.end_date) # job end date
print(job.source_records) # how many records were applied
print(job.processed_records) # how many records were processed
print(job.price) # price for processed records
```
#### Create new job
```
input_data = '''"ID","ADRES"
6876,"34-404, PYZÓWKA, PODHALAŃSKA 100"
'''
job_config = JobConfig('my job')
job_config.input_format(field_separator=',', text_delimiter='"')
job_config.input_column(0, name='ID', function='PRZEPISZ')
job_config.input_column(1, name='ADRES', function='DANE_OGOLNE')
job_config.module_std(address=1)
job_config.extend(gus=True,
geocode=True,
teryt=False,
building_info=False,
diagnostic=False,
area_characteristic=False,
financial_scoring=False)
job = dq.submit_job(job_config, input_data=input_data) # with data in a variable
job = dq.submit_job(job_config, input_file='my_file.csv') # with data inside file
print(job.id)
print(job.name)
print(job.status)
...
```
#### Create new deduplication job
```
input_data = '''unikalne_id;imie_i_nazwisko;kod_pocztowy;miejscowosc;adres;email;tel;CrmContactNumber;data
1;Jan Kowalski;37-611;Cieszanów ;Dachnów 189;abc@wp.pl;605936000;abc123;2017-11-08 12:00:00.000
2;Adam Mickiewicz Longchamps de Berier;66-400;Gorzów Wlkp.;Widok 24;qqq@ft.com;48602567000;a2b2c2;2017-11-08 12:00:00.000
3;Barbara Łęcka;76-200;Słupsk;Banacha 7;bb@gazeta.pl;79174000;emc2;2017-11-08 12:00:00.000
4;KAROL NOWAK;22-122;LEŚNIOWICE;RAKOLUPY DU—E 55;kn@ll.pp;0;f112358;2017-11-08 12:00:00.000
5;Anna Maria Jopek;34-722;Podwilk;Podwilk 464;amj@gmail.com;606394000;eipi10;2017-11-08 12:00:00.000
6;Mariusz Robert;37-611;Cieszanów ;Dachnów 189;abc@wp.pl;605936000;abc123;2017-11-08 12:00:00.000
'''
job_config = JobConfig('pr2')
job_config.input_format(field_separator=';', text_delimiter='"')
job_config.input_column(0, name='unikalne_id', function='ID_REKORDU')
job_config.input_column(1, name='imie_i_nazwisko', function='IMIE_I_NAZWISKO')
job_config.input_column(2, name='kod_pocztowy', function='KOD_POCZTOWY')
job_config.input_column(3, name='miejscowosc', function='MIEJSCOWOSC')
job_config.input_column(4, name='adres', function='ULICA_NUMER_DOMU_I_MIESZKANIA')
job_config.input_column(5, name='email', function='EMAIL1')
job_config.input_column(6, name='tel', function='TELEFON1')
job_config.input_column(7, name='CrmContactNumber', function='PRZEPISZ')
job_config.input_column(8, name='data', function='CZAS_AKTUALIZACJI')
job_config.deduplication(on=True)
job_config.module_std(address=True, names=True, contact=True)
job_config.extend(gus=True, geocode=True, diagnostic=True)
job = dq.submit_job(job_config, input_data=input_data)
print(job)
...
```
Available column functions:
* addresses
* KOD\_POCZTOWY
* MIEJSCOWOSC
* ULICA\_NUMER\_DOMU\_I\_MIESZKANIA
* ULICA
* NUMER\_DOMU
* NUMER\_MIESZKANIA
* NUMER\_DOMU\_I\_MIESZKANIA
* WOJEWODZTWO
* POWIAT
* GMINA
* names
* IMIE
* NAZWISKO
* NAZWA\_PODMIOTU
* IMIE\_I\_NAZWISKO
* people/companies
* PESEL
* NIP
* REGON
* contact
* EMAIL1
* EMAIL2
* TELEFON1
* TELEFON2
* dates
* DATA\_URODZENIA
* CZAS\_AKTUALIZACJI
* mixed
* DANE\_OGOLNE
* id
* ID\_REKORDU
* others
* PRZEPISZ
* POMIN
To process input columns, you must enable the corresponding module. Method module\_std is used to set active modules:
* address
* names
* contact
* id\_numbers
For address module to be started it is necessary to ensure at least one column with the role listed below:
* DANE\_OGOLNE
* KOD\_POCZTOWY
* MIEJSCOWOSC
Analogously for other modules:
* names require one of
* DANE\_OGOLNE
* IMIE
* NAZWISKO
* IMIE\_I\_NAZWISKO
* NAZWA\_PODMIOTU
* contact
* DANE\_OGOLNE
* EMAIL1
* EMAIL2
* TELEFON1
* TELEFON2
* id
* DANE\_OGOLNE
* PESEL
* NIP
* REGON
#### Check job state
```
state = dq.job_state('3f14e25e-9f6d-41ff-a4cb-942743a37b73') # input parameter: job id
print(state) # 'WAITING' or 'FINISHED'
```
#### Cancel job
```
dq.cancel_job('3f14e25e-9f6d-41ff-a4cb-942743a37b73') # input parameter: job id
```
#### Retrieve job report
```
report = dq.job_report('3f14e25e-9f6d-41ff-a4cb-942743a37b73') # input parameter: job id
print(report.quality_issues)
print(report.quality_names)
print(report.results)
```
#### Save job results
```
dq.job_results('3f14e25e-9f6d-41ff-a4cb-942743a37b73', 'output.csv')
```
#### Delete job and its results
```
dq.delete_job('3f14e25e-9f6d-41ff-a4cb-942743a37b73') # input parameter: job id
```
## Contributing
Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.
You can contribute in many ways:
### Types of Contributions
#### Report Bugs
Report bugs at .
If you are reporting a bug, please include:
* Your operating system name and version.
* Any details about your local setup that might be helpful in troubleshooting.
* Detailed steps to reproduce the bug.
#### Fix Bugs
Look through the GitHub issues for bugs. Anything tagged with “bug” and “help wanted” is open to whoever wants to implement it.
#### Implement Features
Look through the GitHub issues for features. Anything tagged with “enhancement” and “help wanted” is open to whoever wants to implement it.
#### Write Documentation
Python client for dataquality.pl could always use more documentation, whether as part of the official Python client for dataquality.pl docs, in docstrings, or even on the web in blog posts, articles, and such.
#### Submit Feedback
The best way to send feedback is to file an issue at .
If you are proposing a feature:
* Explain in detail how it would work.
* Keep the scope as narrow as possible, to make it easier to implement.
* Remember that this is a volunteer-driven project, and that contributions are welcome :)
### Get Started!
Ready to contribute? Here’s how to set up *dq-batch-client* for local development.
1. Fork the *dq\_batch\_client* repo on GitHub.
2. Clone your fork locally:
```
$ git clone git@github.com:your_name_here/dq_batch_client.git
```
3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:
```
$ mkvirtualenv dq_batch_client
$ cd dq_batch_client/
$ python setup.py develop
```
4. Create a branch for local development:
```
$ git checkout -b name-of-your-bugfix-or-feature
```
Now you can make your changes locally.
5. When you’re done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:
```
$ flake8 dq tests
$ python setup.py test or pytest
$ tox
```
To get flake8 and tox, just pip install them into your virtualenv.
6. Commit your changes and push your branch to GitHub:
```
$ git add .
$ git commit -m "Your detailed description of your changes."
$ git push origin name-of-your-bugfix-or-feature
```
7. Submit a pull request through the GitHub website.
### Pull Request Guidelines
Before you submit a pull request, check that it meets these guidelines:
1. The pull request should include tests.
2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst.
3. The pull request should work for Python versions 3.6 - 3.13, and for PyPy. Check and make sure that the tests pass for all supported Python versions.
### Tips
To run a subset of tests:
```
$ pytest tests.test_dq_batch_client
```
## Credits Algolytics Team
* Mateusz Białek <[mateusz.bialek@algolytics.pl](mailto:mateusz.bialek%40algolytics.pl)>
* Łukasz Szpak <[lukasz.szpak@algolytics.pl](mailto:lukasz.szpak%40algolytics.pl)>
## History
### 0.6.0 (2024-12-11)
* RENAME PACKAGE NAME TO dq-batch-client
* Update dependencies: minimal version of Python is now 3.6
* Add financial scoring option to JobConfig
* Update documentation
### 0.5.0 (2018-07-13)
### 0.4.0 (2018-07-10)
### 0.3.0 (2018-07-04)
### 0.2.0 (2018-03-05)
### 0.1.0 (2016-10-19)
* First release on PyPI.
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://algolytics-technologies.gitbook.io/algolytics/algolytics-apis/dq-for-python-api.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.