music-kraken
/
music-kraken-core
3
2
Fork 0
Code Issues 10 Pull Requests Packages Projects 2 Releases Wiki Activity

updated documentation

This commit is contained in:
Hellow 2023-06-23 13:31:49 +02:00
parent d217e550fe
commit b15002b131
3 changed files with 104 additions and 154 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

163
README.md
View File

@ -1,21 +1,20 @@
# Music Kraken
# Music Kraken
<img src="assets/logo.svg" width=300 alt="music kraken logo"/>
- [Music Kraken](#music-kraken)
- [Installation](#installation)
- [Dependencies](#dependencies)
- [Notes for Python 3.9](#notes-for-python-39)
- [Notes for WSL](#notes-for-wsl)
- [Quick-Guide](#quick-guide)
- [CONTRIBUTE](#contribute)
- [Matrix Space](#matrix-space)
- [TODO till the next release](#todo-till-the-next-release)
- [Programming Interface / Use as Library](#programming-interface--use-as-library)
- [Quick Overview](#quick-overview)
- [Data Model](#data-model)
- [Data Objects](#data-objects)
- [Creation](#creation)
- [Appending and Merging data](#appending-and-merging-data)
---
@ -38,14 +37,8 @@ music-kraken
You will need to install these two programms.
- ffmpeg
- pandoc
### Notes for Python 3.9
Unfortunately I use features that newly git introduced in [Python 3.10](https://docs.python.org/3/library/types.html#types.UnionType).
So unfortunately you **CAN'T** run this programm with python 3.9. [#10][i10]
### Notes for WSL
If you choose to run it in WSL, make sure ` ~/.local/bin` is added to your `$PATH` [#2][i2]
@ -59,6 +52,7 @@ When it drops you into the **shell** 2 main things are important:
1. You search with `s: <query/url>`
2. You choose an option with just the index number of the option
3. You download with `d: <options/url>`, where the options are comma seperated
4. You support me by making a pr, or starring my repo.
Trust me it WILL make sense, once you see it.
@ -67,13 +61,13 @@ Trust me it WILL make sense, once you see it.
The syntax for the query is like really simple.
```
> #a <any artist>
> s: #a <any artist>
searches for the artist <any artist>
> #a <any artist> #r <any releas>
> s: #a <any artist> #r <any releas>
searches for the release (album) <any release> by the artist <any artist>
> #r <any release> Me #t <any track>
> s: #r <any release> Me #t <any track>
searches for the track <any track> from the release <any relaese>
```
@ -87,7 +81,7 @@ LOVE YALL *(except nazis ;-;)*
I am happy about every pull request. To contribute look [here](contribute.md).
## Matrix Space
## Matrix Space
<img align="right" src="assets/element_logo.png" width=100>
@ -95,11 +89,19 @@ I decided against creating a discord server, due to piracy communities get often
**Click [this link](https://matrix.to/#/#music-kraken:matrix.org) _([https://matrix.to/#/#music-kraken:matrix.org](https://matrix.to/#/#music-kraken:matrix.org))_ to join.**
## TODO till the next release
> These Points will most likely be in the changelogs.
- [x] Update the Documentation of the new cli.
- [ ] Migrate away from pandoc, to a more lightweight alternative, that can be installed over PiPY.
- [ ] Update the Documentation of the internal structure. _(could be pushed back one release)_
---
# Programming Interface / Use as Library
This application is $100\%$ centered around Data. Thus the most important thing for working with musik kraken is, to understand how I structured the data.
This application is $100\%$ centered around Data. Thus, the most important thing for working with musik kraken is, to understand how I structured the data.
## Quick Overview
@ -110,7 +112,7 @@ This application is $100\%$ centered around Data. Thus the most important thing
```mermaid
---
title: Quick Overview
title: Quick Overview (outdated)
---
sequenceDiagram
@ -245,139 +247,14 @@ Every Service models Data a bit different, and projecting a one-to-many relation
### Creation
```python
# importing the libraries I build on
from music_kraken import objects
import pycountry
song = objects.Song(
genre="HS Core",
title="Vein Deep in the Solution",
length=666,
isrc="US-S1Z-99-00001",
tracksort=2,
target=[
objects.Target(file="song.mp3", path="example")
],
lyrics_list=[
objects.Lyrics(text="these are some depressive lyrics", language="en"),
objects.Lyrics(text="Dies sind depressive Lyrics", language="de")
],
source_list=[
objects.Source(objects.SourcePages.YOUTUBE, "https://youtu.be/dfnsdajlhkjhsd"),
objects.Source(objects.SourcePages.MUSIFY, "https://ln.topdf.de/Music-Kraken/")
],
album_list=[
objects.Album(
title="One Final Action",
date=objects.ID3Timestamp(year=1986, month=3, day=1),
language=pycountry.languages.get(alpha_2="en"),
label_list=[
objects.Label(name="an album label")
],
source_list=[
objects.Source(objects.SourcePages.ENCYCLOPAEDIA_METALLUM, "https://www.metal-archives.com/albums/I%27m_in_a_Coffin/One_Final_Action/207614")
]
),
],
main_artist_list=[
objects.Artist(
name="I'm in a coffin",
source_list=[
objects.Source(
objects.SourcePages.ENCYCLOPAEDIA_METALLUM,
"https://www.metal-archives.com/bands/I%27m_in_a_Coffin/127727"
)
]
),
objects.Artist(name="some_split_artist")
],
feature_artist_list=[
objects.Artist(
name="Ruffiction",
label_list=[
objects.Label(name="Ruffiction Productions")
]
)
],
)
print(song.option_string)
for album in song.album_collection:
print(album.option_string)
for artist in song.main_artist_collection:
print(artist.option_string)
# needs to be added
```
If you just want to start implementing, then just use the code example, I don't care.
For those who don't want any bugs and use it as intended *(which is recommended, cuz I am only one person so there are defs bugs)* continue reading.
If you just want to start implementing, then just use the code example I provided, I don't care.
For those who don't want any bugs and use it as intended *(which is recommended, cuz I am only one person so there are defs bugs)* continue reading, and read the whole documentation, which may exist in the future xD
## Appending and Merging data
If you want to append for example a Song to an Album, you obviously need to check beforehand if the Song already exists in the Album, and if so, you need to merge their data in one Song object, to not loose any Information.
This is how I solve this problem:
```mermaid
---
title: "Collection.append(music_object: MusicObject)"
---
flowchart TD
exist("""
<b>Check if music_object already exists.</b>
<hr>
Gets all indexing values with <code>music_object.indexing_values</code>.
If any returned value exists in <code>Collection._attribute_to_object_map</code>,
the music_object exists
""")
subgraph merge["Merging"]
_merge("""merges the passed in object in the already
existing whith <code>existing.merge(new)</code>""")
_map("""In case a new source or something simmilar
has been addet, it maps the existing object again.
""")
_merge --> _map
end
subgraph add["Adding"]
__map("""map the values from <code>music_object.indexing_values</code>
to <code>Collection._attribute_to_object_map</code> by writing
those values in the map as keys, and the class I wanna add as values.
""")
_add("""add the new music object to <code>_data</code>""")
__map --> _add
end
exist-->|"if it doesn't exist"|add --> return
exist-->|"if already exists"|merge --> return
```
This is Implemented in [music_kraken.objects.Collection.append()](documentation/objects.md#collection). The merging which is mentioned in the flowchart is explained in the documentation of [DatabaseObject.merge()](documentation/objects.md#databaseobjectmerge).
The <u>indexing values</u> are defined in the superclass [DatabaseObject](documentation/objects.md#databaseobject) and get implemented for each Object seperately. I will just give as example its implementation for the `Song` class:
```python
@property
def indexing_values(self) -> List[Tuple[str, object]]:
return [
('id', self.id),
('title', self.unified_title),
('barcode', self.barcode),
*[('url', source.url) for source in self.source_collection]
]
```
[i10]: https://github.com/HeIIow2/music-downloader/issues/10
[i2]: https://github.com/HeIIow2/music-downloader/issues/2

31
contribute.md
View File

@ -1,18 +1,31 @@
> IMPORTANT NOTE: heavily outdated sorryyyyy
# How to contribute
I am always happy about pull requests.
If something is missing, like attributes for an object feel free to either add it youreself or open an issue :3
If something is missing, like attributes for an object feel free to either add it yourself or open an issue, if you choose to just change it, beware that something may have to change. :3
If you wanna contribute, this is for you. Some options will follow:
So here is a List of what you can do:
## Add an audio/metadata source
1. [implement a new page like e.g. Soundcloud](#add-a-new-page)
The audio and Metadata Sources all inherint the class `Page` in [abstract.py](src/music_kraken/pages/abstract.py)
## Add a new Page
You can create a subclass of this class for for example youtube or musify or whatever.
I documented the function it should have in the docstrings of [abstract.py](src/music_kraken/pages/abstract.py). If you are unsure about how it works, look at either the doccumentation *(will get more detailed soon)* or an [example](src/music_kraken/pages/encyclopaedia_metallum.py). For trying you're class you can make a skript simmilar to [this one](src/metal_archives.py). Make sure it is in the same directory though.
The audio and Metadata Sources all inherit from the class `Page`, which can be found in [abstract.py](src/music_kraken/pages/abstract.py).
> Read the part of the documentation, I already have written.
You can create a subclass of this class for, for example `YouTube` or `Musify` or whatever.
1. Just create a new file with the name `your_page.py`
in the [page module](src/music_kraken/pages).
2. Then you can simply copy the contents of the [preset](src/music_kraken/pages/preset.py) over to your file.
3. All the functions you need to implement, can be found in the [preset](src/music_kraken/pages/preset.py).
### Important notes
- There is no need to check if you for example added a source of a song twice. I do much post-processing to the data you scrape in the page classes. You can see what exactly I do in [abstract.py](src/music_kraken/pages/abstract.py).
- Use the connection class how it is laid out in the preset to make the request. This will take care of retrying requests, rotating proxies, consistent use of tor (if selected in the config). You have:
- `connection.get()`
- `connection.post()`
- Look at the code of the pages I already have implemented. Namely:
- [musify.club](src/music_kraken/pages/musify.py) _(heavily making use of web scraping)_
- [YouTube](src/music_kraken/pages/youtube.py) _(using both invidious and piped)_
- [Metal Archives](src/music_kraken/pages/youtube.py)

64
documentation/objects.md
View File

@ -56,9 +56,69 @@ Function | Explanation
`sort()` | takes the same arguments than `list.sort`, and does the same
`__iter__()` | allows you to use collections e.g. a for loop
## Options
### Appending and Merging data
## Metadata
If you want to append for example a Song to an Album, you obviously need to check beforehand if the Song already exists in the Album, and if so, you need to merge their data in one Song object, to not loose any Information.
This is how I solve this problem:
```mermaid
---
title: "Collection.append(music_object: MusicObject)"
---
flowchart TD
exist("""
<b>Check if music_object already exists.</b>
<hr>
Gets all indexing values with <code>music_object.indexing_values</code>.
If any returned value exists in <code>Collection._attribute_to_object_map</code>,
the music_object exists
""")
subgraph merge["Merging"]
_merge("""merges the passed in object in the already
existing whith <code>existing.merge(new)</code>""")
_map("""In case a new source or something simmilar
has been addet, it maps the existing object again.
""")
_merge --> _map
end
subgraph add["Adding"]
__map("""map the values from <code>music_object.indexing_values</code>
to <code>Collection._attribute_to_object_map</code> by writing
those values in the map as keys, and the class I wanna add as values.
""")
_add("""add the new music object to <code>_data</code>""")
__map --> _add
end
exist-->|"if it doesn't exist"|add --> return
exist-->|"if already exists"|merge --> return
```
This is Implemented in [music_kraken.objects.Collection.append()](documentation/objects.md#collection). The merging which is mentioned in the flowchart is explained in the documentation of [DatabaseObject.merge()](documentation/objects.md#databaseobjectmerge).
The <u>indexing values</u> are defined in the superclass [DatabaseObject](documentation/objects.md#databaseobject) and get implemented for each Object seperately. I will just give as example its implementation for the `Song` class:
```python
@property
def indexing_values(self) -> List[Tuple[str, object]]:
return [
('id', self.id),
('title', self.unified_title),
('barcode', self.barcode),
*[('url', source.url) for source in self.source_collection]
]
```
## Song