Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT]: Icons on a seperate .md file #235

Open
1 task done
hawl1 opened this issue Sep 13, 2024 · 25 comments
Open
1 task done

[FEAT]: Icons on a seperate .md file #235

hawl1 opened this issue Sep 13, 2024 · 25 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed

Comments

@hawl1
Copy link
Contributor

hawl1 commented Sep 13, 2024

Description

The thing is, everytime when I open up the skill icons repo, the entire svgs get loaded from github to here which causes ratelimiting for a minute, if we put the icon list on a seperate markdown file, this issue will be solved (i guess)

Additional Information

No response

Code of Conduct Acknowledgment

  • I have read and agree to the Code of Conduct.
@LelouchFR
Copy link
Owner

@hawl1 The thing is that after-wards, users could not notice it and open more issues with icons that already exists... I don't really know if it's a good idea ATM.

@hawl1
Copy link
Contributor Author

hawl1 commented Sep 13, 2024

@hawl1 The thing is that after-wards, users could not notice it and open more issues with icons that already exists... I don't really know if it's a good idea ATM.

i mean, putting another checkbox that says "i have checked if icon exists or not" on the issue template shouldn't really be causing any issues

@LelouchFR
Copy link
Owner

@hawl1 The thing is that after-wards, users could not notice it and open more issues with icons that already exists... I don't really know if it's a good idea ATM.

i mean, putting another checkbox that says "i have checked if icon exists or not" on the issue template shouldn't really be causing any issues

Yeah I understand that, but even if, it's kinda like the "I have read and agree to the Code of Conduct", people check it but never read it, but yeah it could be a good idea. I would like to know what other people think about it.

@josemateuss @ItsJoshBrown @Francois-lenne @IPdotSetAF @RushilJalal @theMoor9

@theMoor9
Copy link
Contributor

theMoor9 commented Sep 13, 2024

I think removing the list from the readme could lead to redundancies regarding issues as well, and I also believe it’s not the best way to present the project. However, I think a <details> tag for the section of the icons table could be a solution, or at least part of it idk. Though I’m not sure if this would fully resolve the problem or fit the intended style of the repo, if I understood correctly.

@IPdotSetAF
Copy link
Contributor

I'm not a fan of a separate .md file, as it's the best place to showcase the icon set.

instead if we could somehow reduce the number of downloaded images, it would solve the rate limit issue.

@theMoor9
Copy link
Contributor

I need to correct myself

The detail tag may not resolve the issue without js which can reduce the downloaded images as @IPdotSetAF suggested. But is not supported on .md

I think there aren't alternatives other than moving the table elsewhere. It could be splitted in a preview in the readme and the full table on a separate .md but the redundancy issue will remain.

I got no other options.

@IPdotSetAF
Copy link
Contributor

I need to correct myself

The detail tag may not resolve the issue without js which can reduce the downloaded images as @IPdotSetAF suggested. But is not supported on .md

We could use the go server itself to generate the icon sets.

Or we could pre-render them

@theMoor9
Copy link
Contributor

We could use the go server itself to generate the icon sets.

Or we could pre-render them

I don't know, does the .md file support this? It's quite limited from what I happen to know...
Anyway this is beyond my capabilities, i dont have the time to dive in it. I'm sorry :\

@LelouchFR
Copy link
Owner

rendering it from the go server could lead to a possible 5XX error (same than doing =all for the icons) which could be kinda bad ig, because then there could be a possibility that the icons will never show up... the details tag could be good, but I don't really know if they get rendered before clicking on it or if there is a way to make it render on click of the details, I'm going to read the doc about it.

@ItsJoshBrown
Copy link
Contributor

I'm not certain about this. At one point, I had thought displaying the most popular Icons on the README and adding the rest to a website with a search feature or something of the sorts. However, I agree with @LelouchFR that if there’s no way for people to find what they’re looking for in some way, they might not bother searching for it at all.

There are things like Actions that you can run against an issue when it's raised to grep a file name or something, I assume this would be hit or miss though depending if someone followed the issue Template to a T. and probably would just fail if multiple icons were listed in the opened issue.

@hawl1
Copy link
Contributor Author

hawl1 commented Sep 14, 2024

I think about hosting the svg icon files on a seperate place like cloudflare pages so this way github doesn't ratelimit us

this repo has bunch of icons which gets loaded without any trouble

@hawl1
Copy link
Contributor Author

hawl1 commented Sep 14, 2024

We could use the go server itself to generate the icon sets.

Or we could pre-render them

The vercel quota will skyrocket if you do that

@LelouchFR
Copy link
Owner

When put in a details tag, they get rendered after having clicked on the summary from what I've experienced by checking the network. @hawl1 can you say us your rate limit, I don't know if we have the same because I don't seem to have a problem at all rendering all the icons, just to check.

do this command (changing the YOUR-OAUTH-TOKEN by your token ofc) to check your rate limit:

curl -H "Authorization: token YOUR-OAUTH-TOKEN" -X GET https://api.github.com/rate_limit

I'm just thinking about how to convert the readme-format script to make out some html instead of a md table

@LelouchFR LelouchFR added documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed labels Sep 14, 2024
@hawl1
Copy link
Contributor Author

hawl1 commented Sep 14, 2024

When put in a details tag, they get rendered after having clicked on the summary from what I've experienced by checking the network. @hawl1 can you say us your rate limit, I don't know if we have the same because I don't seem to have a problem at all rendering all the icons, just to check.

do this command (changing the YOUR-OAUTH-TOKEN by your token ofc) to check your rate limit:

curl -H "Authorization: token YOUR-OAUTH-TOKEN" -X GET https://api.github.com/rate_limit

I'm just thinking about how to convert the readme-format script to make out some html instead of a md table

here you go

{
  "resources": {
    "core": {
      "limit": 60,
      "remaining": 59,
      "reset": 1726324001,
      "used": 1,
      "resource": "core"
    },
    "graphql": {
      "limit": 0,
      "remaining": 0,
      "reset": 1726324285,
      "used": 0,
      "resource": "graphql"
    },
    "integration_manifest": {
      "limit": 5000,
      "remaining": 5000,
      "reset": 1726324285,
      "used": 0,
      "resource": "integration_manifest"
    },
    "search": {
      "limit": 10,
      "remaining": 10,
      "reset": 1726320745,
      "used": 0,
      "resource": "search"
    }
  },
  "rate": {
    "limit": 60,
    "remaining": 59,
    "reset": 1726324001,
    "used": 1,
    "resource": "core"
  }
}

@ItsJoshBrown
Copy link
Contributor

@LelouchFR you can do something like this. althoguht I'm not really that great with Shell most of this was me googling how to or using kind of what you had already done. so I really am not sure how to make it POSIX compliant. so if you just add these on the README.md where you want the icon table to show up it should work.

<!-- START ICONS LIST -->
<!-- END ICONS LIST -->

Then here you would mark where end of the rest of the README.md so it can be added before the HTML table.

head -96 README.md > README.tmp && mv README.tmp README.md

This still keeps the images on GH so not sure if that will change anything for anyone. Is it the actual rate limit that's being reached or is it just the images loading slow? I'm not really sure how to check if a rate limit was reached, although I'm not using the api so maybe that's why?

!/bin/bash

get_maximum_length() {
    items="$1"
    maximum=0
    for item in $items; do
        if [[ ${#item} -gt $maximum ]]; then
            maximum=${#item}
        fi
    done
    echo "$maximum"
}

# Start and end markers from README.md

START_MARKER="<!-- START ICONS LIST -->"
END_MARKER="<!-- END ICONS LIST -->"

head -96 README.md > README.tmp && mv README.tmp README.md

mapfile -t icon_list < <(find assets -maxdepth 1 -type f -name '*.svg' ! -name '*-light.svg' ! -name '*-dark.svg' | sort)

icons_counter="${#icon_list[@]}"

columns=4

rows=$(((icons_counter + columns - 1) / columns))

table_headers="<tr>"
for ((i = 0; i < columns; i++)); do
    table_headers+="<th>Icon ID</th><th>Icon</th>"
done
table_headers+="</tr>"

icon_rows=""
for ((row = 0; row < rows; row++)); do
    icon_rows+="<tr>"
    for ((col = 0; col < columns; col++)); do
        index=$((row + col * rows))
        if [[ index -lt icons_counter ]]; then
            icon_id=$(basename "${icon_list[$index]}" | sed 's/\.[^.]*$//; s/-auto//g')
            img_tag="<img src=\"${icon_list[$index]}\" width=\"48\">"
            icon_rows+="<td>$icon_id</td><td>$img_tag</td>"
        else
            icon_rows+="<td></td><td></td>"
        fi
    done
    icon_rows+="</tr>"
done

{
  sed -n "/$START_MARKER/q;p" README.md
  
  echo "$START_MARKER"
  echo "<table>"
  echo "$table_headers"
  echo "$icon_rows"
  echo "</table>"
  echo "$END_MARKER"
  
  sed -n "/$END_MARKER/,\$p" README.md
} > README.tmp

mv README.tmp README.md

@LelouchFR
Copy link
Owner

@ItsJoshBrown I'm going to look forward into that.
@hawl1 it's strange, because I'm getting an other input for this, it's strange that you don't have any graphql req...

{
  "resources": {
    "core": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1726428346
    },
    "search": {
      "limit": 30,
      "used": 0,
      "remaining": 30,
      "reset": 1726424806
    },
    "graphql": {
      "limit": 5000,
      "used": 16,
      "remaining": 4984,
      "reset": 1726425069
    },
    "integration_manifest": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1726428346
    },
    "code_search": {
      "limit": 10,
      "used": 0,
      "remaining": 10,
      "reset": 1726424806
    }
  },
  "rate": {
    "limit": 5000,
    "used": 0,
    "remaining": 5000,
    "reset": 1726428346
  }
}

@hawl1
Copy link
Contributor Author

hawl1 commented Sep 16, 2024

@ItsJoshBrown I'm going to look forward into that. @hawl1 it's strange, because I'm getting an other input for this, it's strange that you don't have any graphql req...

{
  "resources": {
    "core": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1726428346
    },
    "search": {
      "limit": 30,
      "used": 0,
      "remaining": 30,
      "reset": 1726424806
    },
    "graphql": {
      "limit": 5000,
      "used": 16,
      "remaining": 4984,
      "reset": 1726425069
    },
    "integration_manifest": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1726428346
    },
    "code_search": {
      "limit": 10,
      "used": 0,
      "remaining": 10,
      "reset": 1726424806
    }
  },
  "rate": {
    "limit": 5000,
    "used": 0,
    "remaining": 5000,
    "reset": 1726428346
  }
}

i did not created a token to do it, instead, i just went it through my browser, probably thats why.

{
  "resources": {
    "core": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1726324193
    },
    "search": {
      "limit": 30,
      "used": 0,
      "remaining": 30,
      "reset": 1726320653
    },
    "graphql": {
      "limit": 5000,
      "used": 4,
      "remaining": 4996,
      "reset": 1726322998
    },
    "integration_manifest": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset": 1726324193
    },
    "source_import": {
      "limit": 100,
      "used": 0,
      "remaining": 100,
      "reset": 1726320653
    },
    "code_scanning_upload": {
      "limit": 1000,
      "used": 0,
      "remaining": 1000,
      "reset": 1726324193
    },
    "actions_runner_registration": {
      "limit": 10000,
      "used": 0,
      "remaining": 10000,
      "reset": 1726324193
    },
    "scim": {
      "limit": 15000,
      "used": 0,
      "remaining": 15000,
      "reset": 1726324193
    },
    "dependency_snapshots": {
      "limit": 100,
      "used": 0,
      "remaining": 100,
      "reset": 1726320653
    },
    "audit_log": {
      "limit": 1750,
      "used": 0,
      "remaining": 1750,
      "reset": 1726324193
    },
    "audit_log_streaming": {
      "limit": 15,
      "used": 0,
      "remaining": 15,
      "reset": 1726324193
    },
    "code_search": {
      "limit": 10,
      "used": 0,
      "remaining": 10,
      "reset": 1726320653
    }
  },
  "rate": {
    "limit": 5000,
    "used": 0,
    "remaining": 5000,
    "reset": 1726324193
  }
}

here is the one with token

@LelouchFR
Copy link
Owner

I tried using a details tag to check if it fetches them before rendering and it seems that it does, so it may still rate limit... maybe opening a discussion about it on the community discussions could be a good idea

@theMoor9
Copy link
Contributor

What about using an optimizer for the SVGs or convert them into PNGs to be displayed? I don't have any clues about how optimized SVGs or PNGs impact the rate, so... What do you think?

@LelouchFR
Copy link
Owner

What about using an optimizer for the SVGs or convert them into PNGs to be displayed? I don't have any clues about how optimized SVGs or PNGs impact the rate, so... What do you think?

Moving them to be png, would not make them automatically change to the user theme.

Normally svg are very easy to render and ask a lot less strength than for png/hard images

@theMoor9
Copy link
Contributor

theMoor9 commented Sep 22, 2024

What about this? SVGO I don't know how much could help tho

@hawl1
Copy link
Contributor Author

hawl1 commented Sep 24, 2024

I think about hosting the svg icon files on a seperate place like cloudflare pages so this way github doesn't ratelimit us

this repo has bunch of icons which gets loaded without any trouble

reminder ^^

@LelouchFR
Copy link
Owner

I think about hosting the svg icon files on a seperate place like cloudflare pages so this way github doesn't ratelimit us
this repo has bunch of icons which gets loaded without any trouble

reminder ^^

yeah, but at this point, it's just hosting another instance of the api on another link, and it's just going to cost... Instead, using the api directly on the readme is going to explode the numbers and do the same thing than hosting it somewhere else.

I just think that the problem is comming from:

  • your internet (not telling that it's bad, but that it cannot handle a lot of img fetches (I'm guessing))
  • github rate-limits you for whatever reason...

else idk what could be the problem.

maybe opening a discussion about it on the community discussions could be a good idea

@IPdotSetAF
Copy link
Contributor

githubs rate limiting is different based on countries/dns setting or vpn/proxy usage.

@hawl1
Copy link
Contributor Author

hawl1 commented Sep 24, 2024

I think about hosting the svg icon files on a seperate place like cloudflare pages so this way github doesn't ratelimit us
this repo has bunch of icons which gets loaded without any trouble

reminder ^^

yeah, but at this point, it's just hosting another instance of the api on another link, and it's just going to cost... Instead, using the api directly on the readme is going to explode the numbers and do the same thing than hosting it somewhere else.

I just think that the problem is comming from:

  • your internet (not telling that it's bad, but that it cannot handle a lot of img fetches (I'm guessing))
  • github rate-limits you for whatever reason...

else idk what could be the problem.

maybe opening a discussion about it on the community discussions could be a good idea

Cloudflare pages are free.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

5 participants